Pulsar EventTime: Transitioning from UTC Epochs to Timezone-Aware Timestamps

Answer to the Exact Question

Is there a documented path toward a native timezone‑aware timestamp type in Pulsar 2.11.x? No. The 2.11.x API exposes only a 64‑bit UTC epoch value for eventTime, and the broker’s internal data model contains no field for a timezone or offset. The official roadmap for 2.11.x and the 2.12.x release notes contain no mention of a future migration to a timezone‑aware type, and the source code for the 2.11.x branch confirms this absence.

Recommended approach for maintaining regional date integrity without ad‑hoc properties? Use an explicit contract in your application layer: encode the desired local time in the message payload (e.g., an ISO‑8601 string with offset) or, if you prefer to keep the timestamp separate, add a dedicated message property such as eventTimeZone. Then perform any timezone conversion in a Pulsar Function or in the consumer before applying windowing logic. This keeps the broker agnostic while guaranteeing consistent interpretation across all consumers.

Practical Steps for Pulsar 2.11.x

1. Define a Schema (Optional but Recommended)
   • Create a simple JSON or Avro schema that includes two fields: eventTimeUtc (long) and eventTimeZone (string).
   • Publish messages using this schema so producers and consumers share a contract.
2. Set the Event Time in UTC

   producer.newMessage()
     .eventTime(System.currentTimeMillis())
     .property("eventTimeZone", "America/New_York")
     .value(payload)
     .send();
   

3. Deploy a Pulsar Function to Normalise Time
   • Write a function that receives the message, reads eventTimeUtc and eventTimeZone, converts the UTC epoch to a ZonedDateTime in the specified zone, and writes the result back as a new field (e.g., eventTimeLocal). The function can also emit the message to a “normalized” topic for downstream consumers.
   • Example Java skeleton:

   public class TimezoneNormalizer implements Function<Message<MySchema>, Message<MySchema>> {
       @Override
       public Message<MySchema> process(Message<MySchema> msg, Context ctx) {
           long utcMillis = msg.getEventTimeUtc();
           String zoneId = msg.getEventTimeZone();
           ZonedDateTime local = Instant.ofEpochMilli(utcMillis)
                                         .atZone(ZoneId.of(zoneId));
           MySchema newPayload = new MySchema(utcMillis, zoneId, local.toEpochSecond());
           return ctx.newMessage(newPayload);
       }
   }
   

4. Consume with Local‑Time Windowing
   • In the consumer, use the eventTimeLocal field for windowing, aggregation, or any time‑based logic. Because the field is now a simple epoch value in the local zone, the consumer can safely apply standard windowing functions.
   • Alternatively, let the consumer perform the conversion on the fly if you prefer to avoid a separate topic.
5. Verify Consistency
   • Publish a few test messages with known UTC timestamps and time zones.
   • Consume from the normalized topic and check that eventTimeLocal matches the expected local epoch (e.g., 2026‑10‑19T17:16:44-04:00 for America/New_York).
   • Use Pulsar CLI pulsar-client or a simple Java consumer to print the fields and confirm alignment.

Why This Works

• The broker never interprets the eventTimeZone property; it merely stores it as metadata.
• All timezone logic lives in the function or consumer, ensuring that every downstream system sees the same local time representation.
• Because the function writes a deterministic eventTimeLocal, windowing engines that rely on epoch values can operate without modification.

What If You Need Broker‑Side Windowing?

Apache Pulsar does not expose a windowing API that understands local calendars. The only way to get broker‑side windowing that respects a specific zone is to pre‑convert the timestamp before the message reaches the broker (e.g., by publishing the local epoch instead of UTC). However, this sacrifices the ability to reconstruct the original UTC instant, so it is generally recommended to keep UTC and perform conversion downstream.

Ask for Missing Diagnostic Detail

To tailor the function to your exact needs, could you confirm whether you have Pulsar Functions enabled in your cluster and whether you prefer to emit the local epoch to a separate topic or keep it within the same message payload?

Verification Checklist

Step	What to Check
Schema Deployment	Producer and consumer both compile against the same schema.
Event Time Setting	Broker shows eventTime as UTC epoch.
Property Storage	Broker retains eventTimeZone property unchanged.
Function Output	Normalized topic contains eventTimeLocal matching expected local epoch.
Consumer Windowing	Aggregations based on eventTimeLocal produce correct results.