Does YugabyteDB Need a Compensating Transaction for Partially Applied Online Schema Changes?
27.3K reputation · 20 Aug 2024, 03:33 UTC
The goal is to guarantee that a schema change applied via YugabyteDB's online DDL can be safely reverted if the migration has only progressed on a subset of tablets, without risking data loss or inconsistency.
Currently, once a background migration starts altering tablets, there is no built‑in mechanism to roll back those tablets to the previous schema; certain DDL operations such as adding a NOT NULL column with a default or changing a data type are irreversible once the table rewrite begins. This raises an open design question about whether a compensating transaction or manual rollback procedure should be introduced for partially propagated changes.
- Should YugabyteDB provide a manual rollback procedure for tablets that have already migrated?
- Could a compensating transaction be added to revert both metadata and data changes safely?
- What performance and consistency implications would such a rollback mechanism introduce?
yugabytedb1 answer
1 question comment
Use comments to ask for clarification. Post a solution as an answer.
2,220 reputation · 20 Aug 2024, 08:06 UTC
While the distributed transaction ensures the schema version is committed atomically across the cluster, it is important to distinguish between the metadata update and the physical data migration for operations that trigger a table rewrite.
In YugabyteDB (YSQL), certain DDLs use a multi-state transition protocol (inspired by Google F1) to prevent inconsistencies. If a migration is interrupted, the system relies on these intermediate states (e.g., delete-only or write-only) to maintain safety. However, for operations that physically rewrite the underlying SSTables, the "rollback" is typically a matter of the cluster ignoring the partially written new data once the schema version is reverted or aborted.
To verify the current state during a suspected hang, users should check the yb-master logs for the specific schema version transition. If the version has not incremented, the physical changes on the tablets are effectively orphaned and will be cleaned up by the system's background compaction processes.