Schema validation failed after downgrade – can broker auto‑rollback topic schema version?
0 reputation · 18 Aug 2022, 14:16 UTC
0 reputation · 18 Aug 2022, 14:16 UTC
The goal is to make a Pulsar broker automatically revert a topic’s schema version when a downgrade is required, so that consumers continuing to use the previous schema version can deserialize messages without triggering a DeserializationException.
Since the current implementation lacks an automatic rollback of the schema version after a failed upgrade, the documented error "Schema validation failed: expected version X, found Y" may cause message loss, and older Pulsar releases (<2.8) have no built‑in rollback capability, while schema propagation to all brokers can be asynchronous, creating inconsistent behavior during testing. Can the broker be configured to auto‑rollback the topic schema version on detection of the validation error? Does the DeserializationException guarantee message loss if uncaught? What verification steps confirm the schema version metadata reflects the rolled‑back state?
35775 reputation · 18 Aug 2022, 22:18 UTC
No, the Pulsar broker cannot automatically roll back a topic’s schema version when a validation error occurs; it simply rejects the produce request and leaves the schema at the new version.
The broker’s schema handling is immutable after a successful upgrade; on a mismatch it returns “Schema validation failed: expected version X, found Y” and aborts the request. There is no configuration flag or API that triggers a rollback, and older Pulsar releases lack any built‑in rollback capability.
setTopicSchema(topic, oldSchema)).DeserializationException.Query the topic’s schema version with the admin API (getTopicSchema(topic)) and verify that the version field matches the rolled‑back value. Then, publish a message using the older schema and check that the consumer processes it without exception, confirming that message loss is avoided only when the exception is caught.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.