st.cache_data: ttl expiry or manual .clear() for data that updates on a schedule?

Short answer

For data that is refreshed at a fixed, externally‑triggered interval, using ttl on the cached function is usually the most appropriate choice. It lets Streamlit automatically invalidate the entry after the specified number of seconds, keeping the cache fresh without extra code. Manual .clear() is useful only when you need to force an immediate refresh outside the regular interval (e.g., an admin button or a webhook that notifies the app that the dataset has changed). The max_entries setting simply limits how many distinct cache keys can be stored; it does not interfere with TTL or manual clearing, but it can cause older keys to be evicted if the limit is reached.

Why TTL usually wins

• Predictable expiry: If the external job runs every hour, set ttl=3600. The cache will be considered stale after that period, and the next user request will trigger a recomputation automatically.
• Zero boilerplate: No need to add buttons, webhook handlers, or external triggers inside the Streamlit script.
• Graceful degradation: Users who hit the app just before the TTL expires still see the most recent data that was available at the last recomputation. The stale window is bounded by ttl.
• Avoids race conditions: Manual clearing can lead to a brief period where some sessions still use the old cache while others have already refreshed, especially under high concurrency.

When to consider manual .clear()

• Irregular updates: If the external job sometimes runs earlier or later than the scheduled window, you might want to invalidate immediately when the job finishes.
• Admin‑triggered refreshes: A “Refresh now” button that developers or power users click.
• Multiple datasets: If you have several cached functions, you can clear only the one that changed.

How to implement manual clear

@st.cache_data(ttl=3600)
def load_data():
    # read from disk or database
    return pd.read_csv("/path/to/file.csv")

# In the UI
if st.button("Refresh data"):
    load_data.clear()
    st.success("Cache cleared – next call will reload")

Interaction with max_entries

max_entries limits the total number of distinct cache keys. It works independently of ttl and .clear(). If you set max_entries=1 for a single function, the cache will never grow beyond that key, so TTL or clear will still govern when the value is recomputed. If you have many cached functions, a low max_entries may cause older functions to be evicted before their TTL expires, but this does not affect the freshness of the data for a given function.

Recommendation for your scenario

1. Determine the exact interval (e.g., every 3 hours) and set ttl accordingly.
2. Use a short TTL if you can tolerate a slight recomputation cost when the job hasn't run yet, or a longer TTL if you want to reduce recomputation.
3. Add an optional manual refresh button if you need occasional ad‑hoc invalidation.
4. Keep max_entries high enough to avoid unintentional eviction of your cached function.

Missing detail that could change the recommendation

Does the external job always finish at the same time, or can it finish earlier/later than the scheduled window? If the timing is highly variable, a manual .clear() triggered by a webhook from the job might be preferable.