Redis Sorted Sets: Building a Production-Ready Priority Queue Without Race Conditions

The Problem: Concurrent Priority Queue Pops That Lose Work

When multiple workers dequeue from a Redis-based priority queue using separate ZRANGE + ZREM calls, a race condition emerges: two workers can read the same highest-priority item, one removes it, and the other's ZREM fails silently or removes the wrong element. This causes lost work, duplicate processing, or inconsistent state.

Thesis: Atomic Lua Scripts Make ZSET-Based Queues Safe

Redis Sorted Sets (ZSET) excel at maintaining ordered collections, but safe priority queue implementation requires atomic operations. A single Lua script can combine the read and remove steps, guaranteeing that only one worker successfully pops each item.

Why ZSETs Work for Queues

Sorted Sets store members with floating-point scores, automatically maintaining order. Lower scores appear first in ascending queries, making ZADD ideal for priority queues where lower numbers mean higher priority. The underlying skip list and hash table provide O(log N) insertions, deletions, and range queries.

For leaderboards, use ZREVRANGE (descending order) to fetch top users, and ZRANK to find a user's position. Range queries like ZRANGEBYSCORE enable score-based filtering, such as "all players with scores between 1000 and 2000."

Building an Atomic Priority Queue

The key insight: Redis executes Lua scripts atomically, preventing interleaved operations. A pop script reads the lowest-score member, removes it, and returns it—all in one transaction.

Here's a production-ready script for a min-priority queue (lower score = higher priority):

local queue_key = KEYS[1]
local result = redis.call('ZRANGE', queue_key, 0, 0, 'WITHSCORES')
if result[1] then
  redis.call('ZREM', queue_key, result[1])
  return {result[1], result[2]}
end
return false

Invoke it with:

EVALSHA  1 myqueue:tasks

This script returns {member, score} or false if empty. Store the SHA in application code and reload if NOSCRIPT errors occur.

Leaderboard Implementation

A leaderboard uses descending order. To get the top 10 players:

ZREVRANGE leaderboard:players 0 9 WITHSCORES

To find a player's rank:

ZRANK leaderboard:players username

Adding or updating a score:

ZADD leaderboard:players 1500 user123

Removing a player:

ZREM leaderboard:players user123

Trade-Offs and Limitations

Memory overhead: Each element requires ~50 bytes plus member string storage. A queue with 1 million items consumes roughly 50MB+.

Score precision: Scores are double-precision floats. Values exceeding 2^53 may lose integer precision, affecting exact ranking comparisons.

Write contention: High-frequency updates to the same score can cause latency spikes. For write-heavy workloads, consider partitioning by priority buckets or using Redis Streams for append-only workloads.

Verification Steps

1. Test atomicity: Run the pop script from three Redis CLI sessions simultaneously. Verify each returns a different item.
2. Benchmark throughput: Use redis-benchmark to measure ZADD rates at 10K, 100K, and 1M elements. Expect sub-millisecond latencies for typical queue sizes.
3. Check memory: Run MEMORY USAGE myqueue:tasks before and after inserting 100K items to validate growth patterns.

Actionable Next Steps

Implement the atomic pop script in your application's queue consumer. Store script SHAs at startup and handle NOSCRIPT errors by reloading. For leaderboards, combine ZADD with expiration times using EXPIRE to auto-prune inactive users. Monitor queue depth with ZCARD and alert on stuck items that exceed your maximum processing time.