A Minimal Resilient Event Store with Redis Streams: Architecture and Operational Checks

If you need an ordered, append-only record of domain events but do not want to operate Kafka or a dedicated event-store database, Redis Streams provide a compact alternative. The useful takeaway: a single Redis node with Append Only File (AOF) persistence, one stream, and one consumer group can act as a durable event store, provided you enforce memory limits, choose a safe eviction policy, and monitor consumer lag.

Requirements for a minimal event store

Before sizing the design, define what the store must do:

• Ordered append-only log. Events are written once and never mutated; consumers read them in insertion order.
• Parallel consumption. Multiple workers process events concurrently without duplicating the entire log.
• Durability across restarts. A process or machine restart must not silently drop acknowledged events.
• Bounded memory. The store must fail predictably or trim explicitly rather than exhaust host RAM.
• At-least-once delivery. If a worker crashes mid-processing, the event must be redelivered.

The smallest suitable design

The smallest viable deployment is a single Redis node (version 6.0 or newer, where Streams and consumer-group semantics are stable) with:

• One dedicated stream per bounded context, for example orders.
• One consumer group, for example processors.
• AOF persistence enabled with appendfsync everysec.
• maxmemory set to a value below total host RAM, paired with maxmemory-policy noeviction.

This topology keeps the operational surface small: one process, one disk file, and one logical log. It is appropriate for internal tools, moderate event volumes, and teams that already run Redis for caching or session state.

Creating the stream and consumer group

Run the following from any host that can reach the Redis port (default 6379) using redis-cli. If requirepass is configured, authenticate first with AUTH <password> or launch the client with -a <password>; no operating-system privileges are required.

# Append an event; the server returns the entry ID
XADD orders * type created amount 100

# Create the consumer group starting at the end of the stream
XGROUP CREATE orders processors $ MKSTREAM

The MKSTREAM option creates the stream if it does not already exist, which is useful for bootstrapping. The $ ID means the group will only see events added after creation.

Reading and acknowledging

A worker claims a batch of new entries with XREADGROUP and acknowledges them after successful processing:

# Claim up to 10 never-delivered entries, blocking up to 2 seconds
XREADGROUP GROUP processors worker-1 COUNT 10 BLOCK 2000 STREAMS orders >

# After processing, acknowledge using the exact ID returned above
XACK orders processors <message-id>

The > operator requests only entries that have not been delivered to any consumer in the group. If a worker crashes before XACK, the entry remains in the Pending Entries List (PEL) and can be reclaimed with XCLAIM (or XAUTOCLAIM on Redis 6.2 and later).

Trust and data boundaries

Redis executes commands from any client that can authenticate. Treat the network path between your application and Redis as a trust boundary:

• Bind Redis to a private interface and require requirepass or ACL users.
• Use TLS if the traffic crosses an untrusted network.
• Remember that AOF files on disk are not encrypted by Redis; protect them with host-level disk encryption or filesystem permissions.

Within the data model, stream entries are immutable. The only way to remove them is explicit trimming with XADD ... MAXLEN ~ <count> or XTRIM. There is no background garbage collection that silently deletes events unless you configure an eviction policy that applies to streams, which is why noeviction is the safer default for an event store.

Operational checks

Verify the deployment before relying on it.

Version and persistence

# Confirm Redis version (run from the same client host)
redis-cli INFO server | grep redis_version

# Confirm AOF is enabled
redis-cli CONFIG GET appendonly
redis-cli CONFIG GET appendfsync

Expected checks: redis_version should be 6.0 or higher; appendonly should return yes. With appendfsync everysec, a hard OS crash can lose up to one second of writes. If that window is unacceptable, switch to always and measure the throughput cost.

Memory policy

redis-cli CONFIG GET maxmemory
redis-cli CONFIG GET maxmemory-policy

For an event store, avoid allkeys-lru or similar policies that may evict stream entries under pressure. Prefer noeviction so that writes fail loudly when memory is exhausted, and control retention with explicit MAXLEN trimming.

Consumer lag

redis-cli XINFO GROUPS orders

Look at the lag field for the processors group. A rising, unbounded lag means consumers cannot keep up with producers; either add consumers, optimize processing, or accept that the stream is growing faster than it is drained.

Durability smoke test

Add an event, restart the Redis process, and read the stream back with XRANGE orders - +. If the entry is missing, persistence is not working as expected.

Failure modes

Failure	Symptom	Mitigation
No AOF/RDB	Empty stream after restart	Enable appendonly yes and verify with a restart test.
Memory exhaustion	OOM command errors or silent eviction	Set maxmemory and noeviction; trim explicitly.
Worker crash before ack	Entry stuck in PEL	Use XCLAIM or XAUTOCLAIM; make processing idempotent.
Consumer group churn	Duplicate deliveries	Design handlers for at-least-once semantics.

Conditions that change the design

The single-node design is a starting point, not a destination. Move to a more complex topology when:

• Write throughput saturates one node. Redis is single-threaded for command execution; once network or CPU limits are reached, shard streams across a Redis Cluster or adopt a log-native system.
• High availability is required. A single node is a single point of failure. Add Redis Sentinel for automatic failover, or run Redis Cluster with replicas.
• Retention outgrows memory. If you need weeks of history but only gigabytes of RAM, archive acknowledged ranges to object storage and trim the stream.
• Cross-region durability is needed. AOF protects against process loss, not datacenter loss. Use cross-region replication or an external event bus.

Limitations

Redis Streams are not a drop-in Kafka replacement. They lack built-in schema registries, complex stream processing topologies, and multi-datacenter replication out of the box. AOF rewrite can introduce latency spikes, and the single-threaded model means very large batches or slow Lua scripts can block all clients. Treat this design as a pragmatic event store for moderate scale, not an infinitely scalable backbone.

A practical final check: after configuring persistence and creating the group, produce a handful of events, consume and acknowledge them, restart Redis, and confirm both the stream length and the group's last-delivered-id survive. If they do, the minimal store is behaving as intended.