Question
Unsupported file format version error when opening a partially upgraded DuckDB file
Neel Orbit
0 reputation · 26 Oct 2020, 13:59 UTC
113.8K views
Loading…
0 reputation · 26 Oct 2020, 13:59 UTC
When a DuckDB upgrade is interrupted, the database file can contain a mix of old and new page formats, causing later opens with an older binary to fail with an “unsupported file format version” error. The goal is to decide whether DuckDB should instead permit read‑only access to such a file, providing a graceful fallback for inspection or recovery.
Constraints include the absence of an automatic rollback mechanism, the risk that forcing an open could exacerbate corruption, and the limited reliability of PRAGMA integrity_check on partially upgraded files. It remains unclear whether a read‑only mode would be safe enough to be useful or whether it should remain prohibited to avoid accidental data loss.
Should DuckDB allow read‑only opening of a file with a newer format version when accessed by an older binary? What safeguards would be required to prevent further damage if read‑only access were permitted?
DuckDB should not permit read‑only opening of a database that contains a newer internal format version than the binary can understand. Allowing such an open would expose the process to:
Therefore, the safe path is to perform a full upgrade with the exact DuckDB binary that created the file before any other binary can open it.
DuckDB’s page format is tightly coupled to the binary version. A partially upgraded file may contain a mix of page headers, page sizes, and checksum algorithms. Even if the engine is started in read‑only mode, internal mechanisms (e.g., WAL replay, page caching, or background compaction) can still trigger writes to the file. Without a complete upgrade, those writes may corrupt the already inconsistent layout, making recovery impossible.
Back up the file immediately. Copy the database file to a separate location; never modify the original until you are sure the operation is safe.
Determine the file’s internal format version. Run the current DuckDB binary with the --verbose flag to print header details:
duckdbduckdb --verbose mydb.duckdb
Look for a line like File format version: 9 or similar. If the binary refuses to start, the output may be truncated; in that case you can try a previous DuckDB release that matches the error message.
Find the matching DuckDB binary. The format version corresponds to a specific release (e.g., version 0.9.x). Download that exact binary from the official releases page.
Run a full upgrade. Start the matching binary and upgrade the file:
duckdb --upgrade mydb.duckdb
The tool rewrites all pages to the latest format. If the upgrade succeeds, the file will be fully consistent.
Verify integrity. After the upgrade, run:
duckdb mydb.duckdb -c "PRAGMA integrity_check;"
A clean result indicates the file is ready for use with newer binaries.
Open with the current binary. Now you can safely use any newer DuckDB release to read the database.
Until such safeguards are proven reliable, the safest recommendation is to perform a full upgrade with the original binary rather than rely on read‑only access.
If you already know the exact DuckDB version that created the file, you can skip the format‑version detection step and jump straight to the upgrade. Otherwise, please share the output of duckdb --verbose (or the error message you see) so we can pinpoint the required binary.
Use comments to ask for clarification. Post a solution as an answer.
5,990 reputation · 27 Oct 2020, 01:53 UTC
When dealing with an "unsupported file format version" error, it is critical to distinguish between a corrupted file and a version-locked one. If an upgrade was interrupted, the file header may have been updated to a newer version number while the actual data pages remain in the older format.
The primary reason read-only access is often denied is that the DuckDB engine performs a version check during the initial attachment phase. If the binary detects a version higher than its own capability, it halts to prevent the engine from misinterpreting page-level layouts or checksum algorithms that may have changed between releases.
To safely proceed without data loss, you should identify the specific version number used during the interrupted upgrade. You can often find this by inspecting the file header using a hex editor or by matching the binary release notes. Once the matching binary is used, you can complete the upgrade or export the data to Parquet.