Question
RDB versus AOF for Low‑Latency Writes in Redis: Choosing the Right Persistence Mode
Sora Nea
0 reputation · 16 Jul 2020, 15:03 UTC
140K views
Loading…
0 reputation · 16 Jul 2020, 15:03 UTC
Goal: achieve the lowest possible write latency in Redis while maintaining a durability level that tolerates at most a few seconds of data loss.
Understanding the trade‑off is complicated because the latency impact of RDB, AOF, and the hybrid mode varies with the fsync policy (always, everysec, no) and with how often background rewrites are triggered.
Which persistence mode yields the lowest 99th‑percentile write latency under a fsync=everysec policy? Does enabling hybrid AOF reduce latency compared to pure AOF while keeping durability comparable to RDB snapshots? How does the frequency of automatic AOF rewrites affect latency compared to periodic RDB snapshots?
31775 reputation · 17 Jul 2020, 00:33 UTC
To achieve the lowest 99th-percentile (p99) write latency while tolerating a few seconds of data loss, RDB (Redis Database) snapshots are the most performant choice. However, if you require a tighter durability guarantee (approximately one second), AOF with appendfsync everysec is the standard balance, though it introduces higher p99 latency than RDB.
RDB provides the lowest write latency because the main process does not perform disk I/O for every write; instead, it forks a child process to handle the snapshot. In contrast, AOF with everysec requires the main thread to occasionally block if the background fsync call takes too long, leading to the latency spikes observed in p99 metrics.
Enabling hybrid AOF (aof-use-rdb-preamble yes) does not significantly reduce the latency of individual write operations, but it reduces the duration and resource intensity of AOF rewrites. By starting the AOF file with an RDB snapshot and appending only the most recent commands, the rewrite process is faster and less likely to cause the system‑wide I/O saturation that triggers p99 spikes.
Both RDB snapshots and AOF rewrites rely on the fork() system call. The impact on latency differs as follows:
redisfork().auto-aof-rewrite-percentage). Because pure AOF rewrites must reconstruct the entire log from memory, they are generally more resource-intensive than RDB snapshots.To implement a low‑latency configuration with a ~1‑second durability window, use these settings in redis.conf:
# Enable AOF
appendonly yes
# Sync to disk every second (Balance of latency and durability)
appendfsync everysec
# Enable Hybrid mode to optimize rewrite performance
aof-use-rdb-preamble yesYou can verify the p99 latency impact using the built‑in benchmark tool. Compare a pure RDB setup against the Hybrid AOF setup:
# Test p99 latency for SET operations
redis-benchmark -t set -n 100000 -qMissing Diagnostic Detail: What is the total size of your dataset in RAM? If the dataset is very large (e.g., >20GB), the fork() operation itself may become the primary source of p99 latency spikes regardless of the persistence mode chosen.
Use comments to ask for clarification. Post a solution as an answer.
31,775 reputation · 17 Jul 2020, 01:42 UTC
With appendfsync everysec, the main thread only blocks during the fsync of the AOF file, which is usually <1 ms on SSDs. The RDB snapshot’s forked child does its own fsync after writing the snapshot; the parent process is free to accept commands, so the p99 impact is minimal (<5 ms). Hybrid mode behaves like pure AOF for writes, but the background RDB rewrite can still be forced into the main thread when the server runs low on memory, causing a short burst of latency (5–10 ms). Enabling auto-aof-rewrite-incremental-fsync yes splits the rewrite into small chunks, keeping the blocking window under 1 ms per chunk.