Implementing Optimistic Concurrency in DynamoDB with Conditional Writes

The Problem: The Lost Update

In a distributed system, two processes often read the same record, modify it locally, and attempt to save it back to the database. Without a concurrency strategy, the second write simply overwrites the first, leading to a "lost update." This is particularly dangerous for financial balances, inventory counts, or state machines where the new state depends strictly on the previous state.

The most efficient way to solve this in DynamoDB is optimistic concurrency control (OCC). Instead of locking a record, which creates bottlenecks, OCC assumes conflicts are rare and verifies that the record has not changed since it was last read before committing a write.

The Minimal Item Design

To implement OCC, you need a versioning mechanism. The smallest suitable design adds a single numeric attribute to your item to track its state version.

• Primary key: your existing partition key, plus the sort key if the table uses one.
• Version attribute: a numeric value (for example, version) that increments with every successful update. Store it as a number, not a string.
• Payload: the actual data being modified.

Avoid adding the version attribute to Global Secondary Indexes (GSIs) unless you have a specific query requirement to find items of a certain version. Keeping the version local to the base table reduces write capacity costs.

Implementing the Conditional Update

The core of this pattern is the ConditionExpression. When updating an item, you tell DynamoDB to perform the operation only if the version currently stored in the database matches the version you read at the start of your transaction. The version increment must happen inside the same UpdateItem expression so the check and the write remain one atomic server-side operation.

Configuration Example

Assume you are updating a user's account balance. You read the item and find version: 5. To update the balance, you execute an UpdateItem call with the following request:

{
  "TableName": "UserAccounts",
  "Key": { "UserId": { "S": "user_123" } },
  "UpdateExpression": "SET balance = :newBalance, version = version + :inc",
  "ConditionExpression": "version = :expectedVersion",
  "ExpressionAttributeValues": {
    ":newBalance": { "N": "150.00" },
    ":inc": { "N": "1" },
    ":expectedVersion": { "N": "5" }
  }
}

Execution details:

• Where to run: an API call sent via the AWS SDK from your application server or Lambda function.
• Permissions: the IAM role requires dynamodb:UpdateItem on the target table.
• Expected result: if the stored version is still 5, the balance updates and the version becomes 6. If another process already changed the version, DynamoDB returns ConditionalCheckFailedException and the item is left unchanged.
• Risk: if you omit version = version + :inc from the UpdateExpression, subsequent writers can no longer detect interleaved changes.

Trust and Data Boundaries

The trust boundary for this operation is the DynamoDB server. The condition is evaluated atomically on the server side immediately before the write. This eliminates the need for a separate distributed lock manager, which would introduce additional latency and a new point of failure.

However, the read phase of the cycle is a critical boundary. By default, DynamoDB reads are eventually consistent. If your application reads a stale version of the item, the subsequent conditional write will fail, even if no other process is currently writing to it. For state changes that must start from the latest data, use ConsistentRead: true during the initial fetch, accepting the latency and capacity behavior that comes with it.

Operational Checks and Failure Modes

Handling the failure is as important as the write itself. A ConditionalCheckFailedException is a business logic signal, not a system error. Handle it separately from ProvisionedThroughputExceededException, which indicates throttling rather than a conflict.

Handling Conflicts

When a conflict occurs, do not perform a blind retry that repeats the same request. It will fail repeatedly and consume write capacity on every attempt. Instead, follow this loop:

1. Catch ConditionalCheckFailedException.
2. Perform a fresh read of the item to get the current version and current data.
3. Re-apply the business logic to the new data.
4. Attempt the UpdateItem again with the updated version number.
5. Apply exponential backoff if conflicts persist, to avoid amplifying contention on a single item.

Monitoring

Watch the ConditionalCheckFailedRequests metric in CloudWatch alongside throttling metrics. A sustained spike in failed condition checks indicates real contention on specific items, which may signal a need to change the data model.

When to Redesign

Optimistic concurrency works well for low-to-moderate contention. Certain conditions require a shift in architecture:

• Hot item contention: if a single item is updated hundreds of times per second, the retry loop drives up latency and burns capacity on failed writes. Consider write sharding, or aggregating updates in a buffer before flushing them to DynamoDB.
• Multi-item atomicity: if several items must change together or not at all, a single-item conditional write is insufficient. Move to TransactWriteItems, accepting the added latency and write cost compared with single-item conditional updates.
• Invariants spanning items: if the condition depends on the state of other items in the table, you need DynamoDB transactions to enforce it, because a single condition expression can only reference the item being written.

Verification: run two concurrent writers against the same key and assert that only one update succeeds per expected version. Force an incorrect expected version in a test and confirm the SDK surfaces ConditionalCheckFailedException while the stored item is unchanged. After each successful write, read the item back and verify the version incremented exactly once.