functools.lru_cache with cache_clear() or functools.cache: bounded memory versus simpler caching for mutable external state

Question

In a long-running Python process that repeatedly calls a function with identical arguments, the choice between functools.lru_cache(maxsize=128) paired with explicit cache_clear() and functools.cache hinges on two competing constraints: keeping resident memory bounded while ensuring cached results do not outlive changes to external mutable state such as files, database rows, or global variables.

lru_cache with a finite maxsize evicts least-recently-used entries, but it keys only on the arguments passed to the wrapped call and does not detect when those external dependencies change; stale values persist until cache_clear() is invoked, which invalidates every entry and sacrifices the hit rate for unrelated argument combinations that remain valid. functools.cache, available from Python 3.9, behaves like lru_cache(maxsize=None): it offers no maxsize parameter, never evicts entries, and can grow without bound under the same external-change conditions. Neither decorator expires entries by time or invalidates automatically when dependencies change.

Given that cache_clear() is an all-or-nothing operation whose effect on concurrent readers is not fully specified at the application level, and that functools.cache is unavailable before Python 3.9, which documented approach better serves a process where memory must stay bounded but external state changes unpredictably? Should the decision change when the number of distinct argument tuples is provably small, or when invalidation can be coordinated across threads?

Tasadduq Burney · Answer

Choosing bounded‑memory caching with functools.lru_cache versus unbounded functools.cache
When memory must stay bounded and the wrapped function’s arguments may become stale because of external mutable data, lru_cache(maxsize=N) is the documented option that can satisfy both constraints, whereas functools.cache offers no size limit and can grow without bound.
Confirmed facts

lru_cache accepts a maxsize argument that caps the number of stored entries, keeping resident memory bounded.
The decorator keys only on the arguments supplied at call time, so it cannot detect changes to external files, database rows, or global variables.
cache_clear() invalidates every cached entry, which may reduce hit rate if called frequently.
functools.cache (Python 3.9+) behaves like lru_cache(maxsize=None), never evicts entries, and is unavailable before Python 3.9.

Practical steps for this scenario

Determine a reasonable maxsize that accommodates the expected number of distinct argument tuples; if the set is provably small, set maxsize to that count.
Wrap the target function with functools.lru_cache(maxsize=desired_limit).
Identify the moments when external state changes are known (e.g., file modification time, DB row update notification) and invoke cache_clear() only at those points.
If external changes are not reliably detectable, consider re‑wrapping the function with a versioned cache or protecting the clear call with a threading lock to avoid undefined concurrent read behavior.

When the recommendation may shift
If you can reliably coordinate invalidation across threads or have a deterministic way to detect external changes, the need for frequent cache_clear() calls diminishes and the bounded‑memory benefit of lru_cache remains sufficient. Conversely, if the number of distinct argument combinations is tiny, setting maxsize to that count eliminates the need for manual clearing altogether.
Please confirm whether you have a dependable mechanism for detecting external state changes (e.g., file timestamps, DB change events) or if you can coordinate cache clearing across threads, as this detail will affect the optimal caching strategy.

functools.lru_cache with cache_clear() or functools.cache: bounded memory versus simpler caching for mutable external state

1 answer

Choosing bounded‑memory caching with functools.lru_cache versus unbounded functools.cache

Confirmed facts

Practical steps for this scenario

When the recommendation may shift

Your perspective belongs here.

0 question comments