EnableRetryOnFailure and Ambiguous Commits: The Duplicate-Write Gap EF Core Leaves Open
0 reputation · 09 May 2022, 08:56 UTC
0 reputation · 09 May 2022, 08:56 UTC
EF Core's EnableRetryOnFailure execution strategy retries a failed operation as a whole, including explicit transactions. When a transient timeout strikes after the database has actually committed, the retry re-runs the same inserts and updates, producing duplicate rows. The documented escape hatch is IExecutionStrategy.ExecuteInTransaction with a verifySucceeded delegate that tests whether the transaction really committed before a retry proceeds.
The gap is that this predicate is entirely application-defined. EF Core ships no idempotency-key or exactly-once mechanism, and the check runs on the same connection after the failure, so its reliability depends on the isolation level and on whether the error was a timeout rather than a genuine rollback. Since EF Core 5.0, automatic savepoints around SaveChanges inside explicit transactions improve partial-failure recovery but do nothing for the ambiguous-commit case. Side effects outside the database, such as HTTP calls or message publishing inside the delegate, are re-executed on every retry regardless.
For a team wiring retries into a transactional write path, what makes a verifySucceeded check trustworthy when the original failure was a timeout rather than a true rollback? Is a unique index on a client-supplied operation identifier a safer primary defense than the predicate alone? And where should non-database side effects sit so a retry cannot repeat them?
35275 reputation · 09 May 2022, 15:42 UTC
verifySucceeded is trustworthy only when the failure is a pure timeout, the connection stays open, and the isolation level lets you read committed data. It cannot distinguish a timeout-after-commit from a genuine rollback, deadlock, or connection break. A unique index on a client-generated idempotency key is a stronger primary defense because it turns duplicate inserts into constraint violations that the retry strategy treats as failures. Non-database side effects (HTTP calls, message publishes, file writes) must live outside the retry scope—after ExecuteInTransaction returns successfully, or in an outbox table written inside the same transaction and processed later.
The predicate runs on the same connection after the exception. It sees the committed rows iff:
SqlException with number -2 or 1205) after the database committed.If any of those conditions fails—deadlock victim, connection pool eviction, network partition—the delegate finds no rows and may incorrectly allow a retry, producing duplicates. Always inspect the exception type before trusting the predicate.
Add a column OperationId (or IdempotencyKey) with a unique index to the target table. Generate a GUID per logical operation in the application layer and include it in every insert. On a retry after an ambiguous commit:
DbUpdateException wrapping a unique-violation error.maxRetryCount, then aborts.Result: at most one row survives, regardless of whether verifySucceeded misbehaved. The key must be unique per logical operation (not per retry attempt), so generate it before the first SaveChanges and pass it down through the call chain.
Any code that must run exactly once—HTTP requests, message broker publishes, file writes—must not execute inside the retry delegate or the transaction callback. Two safe patterns:
await strategy.ExecuteInTransaction(async (ctx, token) => {
await DoDatabaseWork(ctx, token);
return true; // verifySucceeded reads the written rows
}, verifySucceeded: async (ctx, token) => {
return await ctx.Set<Entity>().AnyAsync(e => e.Id == expectedId, token);
});
// Side effects HERE, after the strategy returns successfully
await PublishEventAsync(eventData);
await CallExternalApiAsync(payload);
Write the side-effect payload to an OutboxMessage table in the same transaction as the business data. A separate background process (or CDC pipeline) reads the outbox and performs the external calls. Retries only replay the database write; the outbox row is inserted once, and the processor handles idempotency on its side.
Id (PK) and OperationId (unique).EnableRetryOnFailure(maxRetryCount: 3, maxDelay: TimeSpan.FromSeconds(5)).WAITFOR DELAY in a trigger or close connection after ExecuteNonQuery). Confirm verifySucceeded sees the row and suppresses retry.verifySucceeded and an HTTP call after ExecuteInTransaction. Verify the HTTP call runs exactly once per successful attempt.verifySucceeded may block or see stale data, making it unreliable even for timeouts.Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.