Boost Spicedb Performance with Built‑In Evaluation Caching – A Practical Guide

Problem: Repeated Policy Lookups Slow Down Your Service

In many micro‑service architectures, a single HTTP request may trigger dozens of policy evaluations. If every evaluation hits the persistent store, the cumulative latency can reach hundreds of milliseconds, especially under heavy load. Spicedb’s built‑in evaluation cache is designed to cut those round‑trips by keeping recent decisions in memory or an external cache, but it’s not enabled by default. The question is: how do we turn it on, measure the benefit, and keep the cache from feeding stale decisions?

What the Cache Does

When CACHE_ENABLED is true, Spicedb stores the result of an evaluation (subject, resource, action, decision, and any relevant metadata) in a key‑value store. Subsequent identical requests hit this cache, bypassing database queries and policy rule evaluation. The cache is keyed on the triple + any context values used in the policy. A TTL (time‑to‑live) expires entries after a configurable period, and the internal event stream invalidates entries immediately when a policy changes.

Key Features

• Zero‑configuration cache (in‑memory) or external (Redis) via the cache_config section.
• Default TTL 30 s, configurable per deployment.
• Cache size defaults to 1 M entries; can be tuned with cache_size.
• Event‑driven invalidation: updates to any policy, role, or subject automatically purge affected cache keys.
• Independent of policy schema – can be added or removed without touching policies.

Enabling the Cache – Quick Start

1. Create a minimal spicedb.yaml:

   store:
     driver: postgres
     connection: "postgres://user:pass@localhost:5432/spicedb?sslmode=disable"
   
   cache:
     enabled: true
     ttl: 30s
     size: 1000000
   

2. Start Spicedb with this config:

   spicedb serve --config spicedb.yaml
   

   Run as a user with the spicedb role; root is fine for local experiments.

3. Insert a simple policy via the CLI:

   spicedb write "allow if obj.type == \"document\" && sub.role == \"editor\""
   

4. Perform 10,000 identical evaluations using spicedb eval or a curl request:

   for i in {1..10000}; do
     spicedb eval \
       --subject user:alice \
       --resource document:123 \
       --action read
   done
   

   Capture the average latency from the CLI output or by querying the metrics endpoint /metrics and filtering spicedb_policy_evaluation_latency_seconds.

5. Repeat the same test with --cache-enabled=false or by removing the cache.enabled line, and compare the latency numbers.

What to Expect

In practice, the cached runs typically show 4‑5× lower average latency for repeated subject‑resource‑action combinations, especially when the TTL is longer than the average time between policy changes. The exact numbers depend on your data size and hardware, but the relative improvement is usually consistent.

Managing Cache Size and TTL – Trade‑Offs

Two knobs dominate cache behavior: size and TTL. A larger cache can hold more decisions, improving hit rates, but it consumes more RAM. A longer TTL increases hit rates but risks serving stale decisions if policies change frequently.

Parameter	Benefit	Risk
Cache Size ↑	Higher hit rate	More memory, possible swap
TTL ↑	Longer validity, fewer invalidations	Increased staleness window
TTL ↓	Fresh decisions	More DB lookups, lower hit rate

For a read‑heavy service where policies rarely change, a TTL of 60 s and a cache size of 2 M entries might be a good starting point. Monitor the spicedb_cache_hit_rate_seconds metric; if it stays above 0.9, you’re likely over‑provisioning memory. If it drops below 0.5, consider increasing size or TTL.

Keeping the Cache Consistent – Invalidation Mechanics

Spicedb’s event stream listens for any policy, role, or subject updates. When such an event occurs, the cache entry for affected subjects is purged immediately, ensuring that subsequent evaluations reflect the new rules. You can verify this by:

• Updating a policy mid‑test: spicedb write "allow if obj.type == \"document\" && sub.role == \"viewer\""
• Observing the spicedb_cache_invalidations_total metric increase.
• Ensuring that the next evaluation returns the updated decision within the TTL window.

When using an external cache like Redis, remember to configure the same event stream listeners; otherwise, stale entries may linger until TTL expires.

Limitations and Caveats

• Memory‑based caching is limited by the host’s RAM; on multi‑tenant hosts, consider a dedicated Redis instance.
• Policy updates that occur faster than the TTL can still cause brief periods of staleness; for mission‑critical services, a very short TTL or manual invalidation via the API may be required.
• The cache is not a replacement for proper audit logging; decisions are still recorded in the audit log even when served from cache.

Actionable Checklist

1. Enable cache.enabled in your config and set an initial TTL (e.g., 30 s).
2. Run a benchmark (10,000 identical evals) with and without the cache; record latency and hit rate.
3. Adjust cache.size and ttl based on your hit‑rate target and available memory.
4. Verify event‑driven invalidation by updating a policy and checking that the next evaluation reflects the change promptly.
5. For large workloads, switch to a Redis backend via cache.type: redis and monitor memory usage on the Redis server.
6. Continuously monitor spicedb_cache_hit_rate_seconds and spicedb_cache_invalidations_total to catch regressions.

By following these steps, you can turn Spicedb’s evaluation cache from a hidden feature into a tangible performance lever, reducing latency for your most frequent access checks while maintaining policy consistency.