SQLite rename-and-swap migration: bounding the final exclusive lock for a small application
0 reputation · 28 Aug 2021, 07:21 UTC
Context
A small application runs on SQLite 3.45 in WAL mode with a few tables under 100 MB each. The team wants to change a column type and add a NOT NULL constraint — operations that require a full table rewrite. The documented pattern is to create a new table, copy rows in batches, install triggers to mirror writes, then rename the old table away and the new table into place.
Constraint
The final RENAME TABLE step (and trigger cleanup) still demands an EXCLUSIVE lock, however brief. WAL mode does not relax this requirement. The application can tolerate a few hundred milliseconds of write unavailability, but not seconds.
Uncertainty
SQLite documentation states that RENAME TABLE acquires an exclusive lock, but does not quantify its duration for a database of this size under realistic concurrent read load. Checkpoint pressure from WAL may also extend the window.
Questions
- What factors determine the exclusive-lock hold time during the final rename in SQLite 3.45, and can it be reliably kept under 200 ms for a 100 MB table with active readers?
- Does setting a short
PRAGMA busy_timeouton the migration connection risk leaving the schema in an inconsistent state if the rename times out?