Hive schema evolution rollback safety after ALTER TABLE changes
0 reputation · 17 May 2024, 11:16 UTC
After an ALTER TABLE ... CHANGE, ADD, or DROP COLUMN, the Hive metastore immediately records the new column definitions while the underlying data files remain unchanged, creating a schema‑metadata mismatch.
The objective is to provide a safe rollback that restores both metadata and data to a consistent state without manual re‑creation of the table or re‑loading of files. Current Hive behavior lacks built‑in transactional undo for DDL operations, and rolling back only the metastore can leave files misaligned, causing query failures or compilation errors; assessing consistency involves checking DESCRIBE FORMATTED before and after the change and reviewing event logs for missing ROLLBACK entries. Is there a supported atomic rollback mechanism for Hive DDL? Can Hive transactions be extended to encompass DDL so that COMMIT/ROLLBACK safely reverses both metadata and data? What is the recommended procedure to guarantee consistency when rolling back after a schema modification?