functools.lru_cache with cache_clear() or functools.cache: bounded memory versus simpler caching for mutable external state
0 reputation · 16 Apr 2023, 21:07 UTC
0 reputation · 16 Apr 2023, 21:07 UTC
In a long-running Python process that repeatedly calls a function with identical arguments, the choice between functools.lru_cache(maxsize=128) paired with explicit cache_clear() and functools.cache hinges on two competing constraints: keeping resident memory bounded while ensuring cached results do not outlive changes to external mutable state such as files, database rows, or global variables.
lru_cache with a finite maxsize evicts least-recently-used entries, but it keys only on the arguments passed to the wrapped call and does not detect when those external dependencies change; stale values persist until cache_clear() is invoked, which invalidates every entry and sacrifices the hit rate for unrelated argument combinations that remain valid. functools.cache, available from Python 3.9, behaves like lru_cache(maxsize=None): it offers no maxsize parameter, never evicts entries, and can grow without bound under the same external-change conditions. Neither decorator expires entries by time or invalidates automatically when dependencies change.
Given that cache_clear() is an all-or-nothing operation whose effect on concurrent readers is not fully specified at the application level, and that functools.cache is unavailable before Python 3.9, which documented approach better serves a process where memory must stay bounded but external state changes unpredictably? Should the decision change when the number of distinct argument tuples is provably small, or when invalidation can be coordinated across threads?
A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.