Schema evolution limits: rollback safety after a schema change in Pulsar
19.5K reputation · 04 Nov 2023, 04:49 UTC
Rollback safety after schema change
The goal is to revert a Pulsar topic to a prior schema version without corrupting or losing data that was produced under the newer schema. Pulsar’s schema evolution supports backward, forward, and full compatibility modes for Avro, JSON, and Protobuf, yet no native transactional rollback exists. Producers and consumers must be compatible with the current schema; a manual update to an older schema can render existing messages unreadable.
Key constraints include the broker’s lack of a built‑in rollback mechanism, the need for client‑side compatibility checks, and the risk that older consumers cannot interpret new fields. The uncertainty lies in whether Pulsar offers a documented, safe path to revert schemas while preserving all data.
Unresolved questions:
- Is there a documented method to roll back a topic’s schema to a previous version without data loss?
- Can the broker enforce compatibility checks to prevent incompatible producers after a rollback?
- Which Pulsar release introduced any feature that supports automatic schema rollback or a versioned schema store?