DuckDB INSERT ON CONFLICT: unclear guarantees when two sessions upsert the same key concurrently
0 reputation · 27 Dec 2021, 12:35 UTC
0 reputation · 27 Dec 2021, 12:35 UTC
I am designing a write path on DuckDB (0.8.0 or newer) where a client may retry an insert after a timeout. The plan is to rely on INSERT ... ON CONFLICT DO NOTHING (or DO UPDATE) against a table with a primary key, so a retried statement cannot create a duplicate row.
The single-session case is clear: with a unique constraint in place, re-issuing the same upsert leaves exactly one row. What I cannot pin down from the documentation is the concurrent case. When two transactions attempt an upsert on the same key at the same time, the statements appear to be serialized, but the exact isolation guarantees are not spelled out — in particular whether ON CONFLICT DO UPDATE can lose an update from the other transaction, and whether a serialization failure can surface to the client instead of a clean conflict resolution.
This matters because I need to know whether the upsert alone makes retries safe, or whether I still need application-level retry logic for deadlock/serialization errors on top of it.
Specifically:
INSERT ... ON CONFLICT in DuckDB atomic with respect to a concurrent upsert on the same key, or can a duplicate-key error or lost update still occur?A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.