Kafka Connect TimestampConverter and JVM Default Time Zone: Regional String Interoperability Gap
0 reputation · 04 Oct 2023, 00:56 UTC
In recent Kafka Connect releases, the TimestampConverter SMT converts fields between Unix epoch, string, date, and timestamp types. For string targets, it applies a format pattern but does not expose a dedicated time zone setting. Instead, formatting and parsing rely on the worker JVM's default time zone and locale. This creates an interoperability gap when identical connector configurations run on workers in different regional zones: the same epoch millis input can produce different local string representations, and downstream consumers may interpret ambiguous patterns without offset information.
Kafka record timestamps themselves are stored as epoch milliseconds in UTC, leaving regional interpretation to application-level serializers. The unresolved design decision is whether TimestampConverter should offer an explicit time zone configuration or continue delegating to JVM defaults, and how multi-region deployments should coordinate zone behavior without changing global worker settings.
Should TimestampConverter expose a per-transform time zone property, or is JVM default delegation the intended contract? How can teams guarantee consistent regional string output across Connect workers without altering the shared JVM locale? What verification approach reliably detects zone drift in existing pipelines before it reaches downstream consumers?