JDBC Driver ↔ ReplicatedMergeTree: Zero‑Downtime Schema Evolution via Online ALTER or Drop‑and‑Swap
0 reputation · 21 Dec 2023, 19:40 UTC
0 reputation · 21 Dec 2023, 19:40 UTC
Upgrade a small Java application’s ClickHouse schema without service interruption. The application uses the official JDBC driver and a ReplicatedMergeTree table that must stay writable during the change.
How the JDBC driver and the application handle the transient lock and rename window, and what impact materialized views have on write amplification during the migration.
31275 reputation · 21 Dec 2023, 20:43 UTC
To achieve zero-downtime schema evolution with a Java application using the official ClickHouse JDBC driver, you should avoid "drop-and-swap" in favor of EXCHANGE TABLES (if supported by your version) or a metadata-only ALTER. The JDBC driver does not provide transparent retries for write-lock exceptions; therefore, any transient failures during DDL execution must be handled by application-level retry logic with exponential backoff.
The ClickHouse JDBC driver operates as a pass-through for SQL execution. It does not automatically detect or retry queries that fail due to server-side write locks or DDL conflicts. If an ALTER TABLE operation triggers a lock that causes an INSERT to fail, the driver will throw a SQLException. To maintain availability, implement a retry wrapper around your data access layer.
A traditional drop-and-swap creates a race condition where the table name is temporarily missing, leading to immediate SELECT failures. While connection pools generally do not cache table names, they may cache Prepared Statements. If a statement is bound to a table that is dropped, it will become invalid.
EXCHANGE TABLES table_old AND table_new. This is an atomic operation that swaps the names of two existing tables, ensuring the application always sees a valid table.maxLifetime on your connection pool (e.g., HikariCP) to cycle connections and clear any driver-level metadata caches.Using a Materialized View (MV) to sync a new schema during migration introduces write amplification proportional to the number of insert blocks. Every block inserted into the source table triggers a corresponding insert into the MV target. This increases:
ADD COLUMN is a metadata-only change. In recent versions, adding a column with a constant DEFAULT is nearly instantaneous and does not rewrite data.table_v2).table_v1 to table_v2, then backfill historical data.EXCHANGE TABLES table_v1 AND table_v2.To refine this recommendation, please provide the ClickHouse server version. Whether an ALTER is treated as a metadata-only change or a heavy mutation depends entirely on the version and the specific column definition.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.