sqlite3_backup WAL snapshot lacks exposed frame number for application-level verification
0 reputation · 28 Dec 2021, 05:24 UTC
0 reputation · 28 Dec 2021, 05:24 UTC
Correlate a backup taken via sqlite3_backup with an application-defined logical point in time (e.g., a committed transaction ID or version marker) so that restored data can be verified against that marker.
The sqlite3_backup API initiates a read transaction on the source database and copies pages through the pager, producing a consistent snapshot that includes all transactions committed up to that read transaction's start. In WAL mode this means the backup captures the main database file and any WAL frames needed for consistency. However, the API does not return or expose the WAL frame number, salt, or transaction ID that identifies the exact snapshot boundary.
Without an exposed frame reference, an application cannot later verify that a restored backup corresponds to a specific committed transaction (for example, the one that wrote a known version row). PRAGMA integrity_check and PRAGMA quick_check only validate internal structural consistency, not completeness relative to an external logical marker.
The documentation notes that the backup API is safe for concurrent use and that a checkpoint before backup is a performance optimization to reduce writer blocking, not a correctness requirement. Alternative approaches such as VACUUM INTO (SQLite 3.27.0+) or file-copy with exclusive locking also do not surface a transaction identifier for cross-referencing.
sqlite3_backup_init?VACUUM INTO or a locked file copy provide any advantage in exposing a snapshot identifier for later verification?No, there is no supported mechanism in the sqlite3_backup API, VACUUM INTO, or standard file-copy methods to retrieve the WAL frame number or transaction ID associated with a snapshot. These identifiers are internal to the SQLite pager and are not exposed to the application layer.
The sqlite3_backup_init function initiates a read transaction on the source database. In WAL mode, this creates a snapshot of the database at a specific point in time. While SQLite uses frame numbers and salts internally to maintain this consistency, the API only exposes page-level progress (via sqlite3_backup_remaining), not the logical transaction boundary. Because the backup process is designed to be transparent to the application, it does not surface the underlying WAL metadata.
| Method | Consistency | Exposes TxID/Frame? | Note |
|---|---|---|---|
sqlite3_backup |
Snapshot | No | Safe for concurrent use. |
VACUUM INTO |
Snapshot | No | Creates a fresh, defragmented file. |
| Locked File Copy | Physical | No | Requires exclusive lock; blocks writers. |
To establish a verifiable link between a backup and a logical point in time, you must embed a marker directly into the database schema. This moves the verification from the transport layer (WAL frames) to the data layer (rows).
sys_backup_marker) with a single row containing a unique identifier, such as a UUID or a monotonically increasing version number.sqlite3_backup_init using the same connection or ensuring no other writer can intervene between the marker update and the backup start.sys_backup_marker table. If the restored value matches your expected marker, the backup is logically complete relative to that transaction.-- 1. Set the marker
BEGIN TRANSACTION;
UPDATE sys_backup_marker SET version_id = 'backup_2026_10_12_01';
COMMIT;
-- 2. Perform sqlite3_backup_init() here
-- 3. Verify after restore
SELECT version_id FROM sys_backup_marker;
-- Expected: 'backup_2026_10_12_01'
This approach assumes you have write access to the source database. If the database is read-only, you cannot use internal markers and must rely on external metadata (such as the timestamp of the last known successful write) which is inherently less precise. Note that PRAGMA integrity_check only confirms that the file is not corrupted; it cannot tell you if a specific transaction was included.
Diagnostic Detail Needed: Are you performing backups on a read-only database handle? If so, the marker-row pattern is not viable, and an alternative synchronization strategy would be required.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.