Using Memcached CAS to Prevent Lost Updates in Read-Modify-Write Workloads

The problem: lost updates in a read‑modify‑write loop

Two web‑request handlers read the same key from Memcached, each adds a new item to a shopping cart locally, and both write the updated cart back with a plain set. The second write overwrites the first, so one addition disappears. This classic race occurs even when Memcached is fast and correct.

How CAS gives an optimistic compare‑and‑swap

When a client fetches an item with the binary protocol, Memcached returns a 64‑bit CAS token together with the value. The token changes whenever the item is replaced, deleted, or expires. A subsequent cas command supplies the token; the server updates the item only if the token still matches the current version. If another client changed the item in the meantime, the token differs and the server replies with a CAS‑mismatch error.

CAS is only available through the binary protocol; ASCII clients cannot request or send a token and will fall back to a non‑atomic get‑then‑set.

When CAS is useful

CAS shines for single‑key read‑modify‑write patterns where lost updates are costly: counters, rate limiters, session flags, or small mutable objects like a shopping cart. It avoids the need for server‑side locking while still guaranteeing that an update is applied only if the underlying value has not changed.

It does not provide multi‑key atomicity or ordering guarantees across different keys.

Worked example: merging a cart with bounded retries

The following pseudocode assumes a binary‑protocol client library that exposes get_with_cas and cas methods (e.g., pylibmc or libmemcached).

# Python‑like pseudocode
token, raw = client.get_with_cas('cart:123')
cart = deserialize(raw)

cart.items.append(new_item)          # local modification
new_raw = serialize(cart)

# Try to store the updated cart; retry a limited number of times
for attempt in range(3):
    result = client.cas('cart:123', new_raw, cas_token=token)
    if result:                       # CAS succeeded
        break
    # CAS mismatch – fetch a fresh token and retry
    token, raw = client.get_with_cas('cart:123')
    cart = deserialize(raw)
    cart.items.append(new_item)      # re‑apply the change
    new_raw = serialize(cart)
else:
    # After retries we could log or fall back to a different strategy
    raise RuntimeError('Unable to update cart after CAS retries')

The loop fetches a fresh token on each failure, guaranteeing that the cas call works with the current version. Bounding the retries prevents latency spikes under high contention.

Trade‑offs and practical limits

• Added client complexity: the caller must handle CAS‑mismatch and implement a retry loop.
• Each failed CAS incurs an extra round‑trip to fetch a new token, increasing latency under contention.
• CAS tokens become stale when the item is evicted, expires, or is overwritten by any other operation, so a high churn rate raises the failure probability.
• Only the binary protocol exposes the token; ASCII‑only clients cannot use CAS.
• There is no transactional guarantee across multiple keys; a operation that touches two separate carts cannot be made atomic with CAS alone.

How to verify CAS works in your environment

1. Start a Memcached instance with the binary protocol enabled (the default port 11211 supports both ASCII and binary). Example command (requires permission to bind to the port): memcached -p 11211 -d.
2. Use a binary‑protocol client library (e.g., pylibmc version 1.6+). Retrieve a key and inspect the returned CAS token; it should be a 64‑bit integer.
3. From two separate processes, read the same key with its token, then each attempt a cas using the original token. One call should succeed, the other should return a CAS‑mismatch.
4. Expire the key (wait for its TTL or issue flush_all) and retry a cas with the old token; the server must reject it.
5. Measure the retry rate under a controlled load (e.g., using memtier_benchmark with a read‑modify‑write script) to see whether the bounded‑retry loop meets your latency budget.

If the tests show CAS mismatches only when another client truly modified the item, you can rely on CAS for safe read‑modify‑write on a single key.

Actionable takeaway

Use Memcached’s CAS token when you need to prevent lost updates on a single key and can tolerate occasional retries. Keep values small, TTLs stable, and restrict yourself to binary‑protocol clients. Implement a bounded retry loop, monitor CAS‑mismatch rates, and fall back to a different coordination mechanism only if retries become too frequent.