Atomic Multi‑Key Ops in Redis with Lua: A Practical Guide

Why Atomicity Matters in Redis

When a single client must update several keys in a consistent way, issuing separate SET or INCR commands is unsafe. Another client could interleave its own commands between yours, leaving the data in a partially‑updated state. In a distributed system this can cause subtle bugs that are hard to reproduce.

Redis offers a built‑in mechanism to guarantee that a group of commands runs as a single, indivisible operation: Lua scripting. Because Lua scripts execute inside the Redis event loop, no other command can run until the script finishes. This guarantees atomicity without the overhead of a distributed transaction protocol.

The Core Idea: A One‑Shot Script

A Lua script is a small chunk of code that Redis evaluates. The script can read and write any key, call Redis commands via the redis.call API, and return a value to the client. When you call EVAL, Redis compiles the script, caches the bytecode, and runs it. Subsequent calls can use EVALSHA to skip recompilation.

Script Size and Limits

• Maximum size: 512 KB. Exceeding this returns ERR Script size exceeded.
• Scripts live in the server’s memory. A single large script can increase the memory footprint, affecting eviction.
• Long‑running scripts block the entire instance, so keep logic tight.

Concrete Example: Atomic Counter with Threshold Check

Imagine a rate‑limiting service that increments a counter for a user and rejects the request if the counter exceeds a threshold. Without scripting, you might run:

INCR user:123:count
GET user:123:count

Between those two commands another client could change the counter, leading to a race. The Lua solution is atomic:

-- counter.lua
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local new = redis.call('INCR', key)
if new > limit then
  return {err = 'limit exceeded'}
end
return new

To load and run this script:

1. Load once and cache: redis-cli --eval counter.lua , 100 – this stores the script in the server’s cache.
2. Later, use EVALSHA to avoid re‑loading: EVALSHA <sha> 1 user:123:count 100.
3. Client permissions: any user with write rights on the key can execute the script.

Expected outcome: the client receives the new counter value if it’s below the limit, or an error message if it’s exceeded. No other client can interleave during the script’s execution.

Testing the Atomicity

Run the script concurrently from multiple terminals:

for i in {1..10}; do redis-cli EVALSHA <sha> 1 user:123:count 100 && echo OK || echo FAIL; done

After all invocations, check the counter:

redis-cli GET user:123:count

The final value should equal the number of successful increments, proving that no race occurred. Use MONITOR to observe that the script runs as a single command in the log.

Trade‑Offs and Limitations

• Blocking: A script runs on the main thread. A 200 ms script will delay all other clients. Keep logic simple.
• Memory Footprint: Caching bytecode increases memory usage. Monitor MEMORY STATS before and after loading scripts.
• Debugging: Errors surface as generic messages. Wrap calls in redis.pcall and add return {err=…} to surface useful diagnostics.
• Version Compatibility: The 512 KB limit applies to Redis 6 and later. Earlier versions had a 128 KB limit.

When to Use Lua vs. MULTI/EXEC

Redis’s MULTI/EXEC pipeline also offers atomicity, but only for a single client’s command sequence. Lua scripts allow multiple clients to invoke the same logic atomically, and can read/write arbitrary keys, including those not known at script load time. Use Lua when:

• You need to perform a read‑modify‑write cycle that spans several keys.
• You want to reduce round‑trip latency by bundling commands.
• Consistency across many clients is critical.

Actionable Checklist for Your Team

1. Identify critical multi‑key operations that need atomicity.
2. Write a minimal Lua script; test locally with EVAL.
3. Load the script once and store its SHA in your application configuration.
4. Replace the original command sequence with EVALSHA calls.
5. Monitor MONITOR or INFO CPU to ensure scripts don’t block.
6. Track memory usage via INFO MEMORY before and after script deployment.
7. Implement error handling in the script to surface meaningful messages.

By following these steps you’ll gain atomicity, reduce network chatter, and keep your Redis deployment healthy.