Redis key-value store: missing stale‑while‑revalidate configuration option
0 reputation · 21 Jul 2020, 03:32 UTC
0 reputation · 21 Jul 2020, 03:32 UTC
Goal: Configure Redis to return the current (stale) value of an expired key while a background process refreshes it, providing a stale‑while‑revalidate behavior without requiring application‑level logic.
Constraints: Redis presently returns nil for expired keys; the lazyfree‑lazy‑expire setting only changes when the key is removed from memory, not what is served on a GET. Implementing SWR manually with Lua scripts or Redis modules adds latency, requires versioning or extra copies of the value, and introduces race‑condition handling such as stampede protection.
Uncertainty: Whether a dedicated configuration directive (e.g., stale‑while‑revalidate‑seconds) could be added to redis.conf to enable this behavior automatically, how it would interact with existing eviction policies and maxmemory limits, and what default TTL for serving stale data would be appropriate.
Questions: Could a redis.conf option be introduced to serve stale values during background refresh? What would be a sensible default for the stale‑serving window? How would this feature coexist with lazyfree‑lazy‑expire and key eviction?
29775 reputation · 21 Jul 2020, 07:14 UTC
Redis has no redis.conf directive that automatically serves a stale value for an expired key while a background refresh runs. The expiration contract is hard‑coded: once the TTL reaches zero, the key is considered non‑existent and GET returns nil. The only configuration that delays the physical removal of the key is lazyfree-lazy‑expire, which only affects when the memory is reclaimed, not what a client receives.
EXPIRE, TTL, and PERSIST.The most common patterns are:
-- Pseudo‑Lua
local val = redis.call('GET', 'mykey')
if val then
return val
end
-- key is expired or missing
-- trigger a background refresh via a separate process
return nil
In this pattern you keep a separate key (e.g., mykey:stale) that holds the last known value. The Lua script first checks mykey; if it’s missing, it falls back to mykey:stale and optionally kicks off a refresh.
Write a small module that exposes a command like SWRGET key. The module can maintain an internal “stale” flag and schedule a background refresh using PROTECTED callbacks. This keeps the logic inside Redis but requires compiling the module and managing its lifetime.
Use SET key value NX PX 300000 to lock refreshes. When a GET sees a nil, it attempts to acquire the lock; if it succeeds, it performs the refresh, otherwise it serves the stale copy.
Enabling lazyfree-lazy‑expire yes only defers the memory reclamation of the key after it has expired. The key is still considered absent to clients. Therefore, the stale‑while‑revalidate logic must store the stale copy in a separate key or in memory managed by a Lua script/module; lazyfree‑lazy‑expire does not help you serve stale data.
maxmemory sooner.expired events will fire only for the original key; the stale copy will remain until you delete it explicitly.There is no “one‑size‑fits‑all” value. A common practice is to set the stale window to a fraction of the original TTL (e.g., 10‑20%) or a fixed value like 30 s, depending on how frequently the background refresh can run and the acceptable staleness for your use case.
To tailor this solution to your environment, please confirm the Redis version you are running:
redis-cli INFO server | grep redis_version
Older releases may not support the same Lua features or module APIs, and the behaviour of lazyfree‑lazy‑expire can vary between 6.x, 7.x, and 8.x.
Redis cannot be configured to return stale values for expired keys out of the box. Implementing stale‑while‑revalidate requires additional logic—Lua scripts, modules, or application‑level coordination—and will affect memory usage and eviction behaviour. Verify your Redis version, choose a stale window that matches your freshness requirements, and decide whether to keep a separate stale copy or handle the logic in Lua.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.