Cache API vs. Durable Objects for Hot-Key Read Coalescing

Question

Managing Tail Latency Under Concurrent Load
When implementing a high-traffic read endpoint in Cloudflare Workers, managing latency spikes during concurrent bursts is critical. Two documented approaches exist for handling "hot keys" to prevent origin overload and reduce tail latency: the Cache API and Durable Objects.

Trade-offs in Coordination
The Cache API allows for fast edge reads and can absorb repeated identical requests at the colo. However, it does not provide strong cross-request serialization, which may lead to cache stampedes where multiple concurrent requests trigger duplicate origin fetches during a miss or eviction.
Conversely, Durable Objects provide a single-threaded execution environment for a specific ID, allowing the developer to collapse duplicate in-flight operations into a single request. While this prevents stampedes, it introduces an additional network hop and creates a potential bottleneck if the QPS for a single object exceeds the isolate's processing capacity.

Scheduling Uncertainty
Because JavaScript execution within a single isolate is single-threaded, CPU-bound work can delay other requests scheduled on that same isolate. It remains unclear how Cloudflare's scheduler distributes bursts of concurrent requests across isolates for a single worker script.

Does the Cache API's lack of serialization consistently result in higher origin load compared to the overhead of a Durable Object hop during a burst?
  Under high concurrency, does the single-threaded nature of a Durable Object introduce more tail latency than the potential for duplicate fetches in the Cache API?

Tasadduq Burney · Answer

Direct Answer
When a single key is requested many times in a short window, the two main Cloudflare Workers mechanisms behave differently:

Cache API serves hits from the edge in sub‑millisecond time, but it has no built‑in coordination. If the key is missing or the entry has expired, every concurrent request that reaches the edge will issue its own origin fetch. This can multiply the origin load proportionally to the number of concurrent misses.
Durable Objects give you a single‑threaded execution environment per ID. The first request for a key that is not already cached can fetch from the origin while all other concurrent requests are queued and receive the same response once it arrives. The cost is a small network hop to the DO’s home colo (typically 1–5 ms) and the fact that the DO processes requests sequentially, which can increase tail latency if the origin is slow or if the DO is busy with CPU‑bound work.

Confirmed Facts (as of 2026‑10‑18)

The Cache API is colocated; reads are served from the edge’s local storage.
Cache API misses trigger an origin request for every concurrent request that reaches the edge.
Durable Objects have a single‑threaded isolate per ID; concurrent requests for the same ID are serialized.
Accessing a Durable Object involves a cross‑colo hop to the object’s home colo, adding 1–5 ms of latency that varies by geography.
Each DO isolate is limited to 128 ms of CPU time per request and 128 MB of memory.

Uncertain or Unverified Aspects

Exact thresholds where one approach outperforms the other (e.g., 100 RPS or 50 ms origin latency) are not backed by published benchmarks and may vary with Cloudflare’s infrastructure changes.
The impact of Cloudflare’s scheduler on how burst traffic is distributed across Worker isolates is not publicly documented.
Cache API’s support for stale-while-revalidate works only when the header is present in the origin response; it does not guarantee stampede protection without careful TTL tuning.

Practical Guidance
Choose the strategy that best matches your traffic profile and the limits you can tolerate:

Low burst, low origin latency – Use the Cache API alone. Set a short max-age and optionally stale-while-revalidate to reduce the chance of a stampede.
High burst or higher origin latency – Use a Durable Object for the miss path. The DO can coalesce the request and then write the result back to the Cache API for subsequent hits.
Very high request rates (thousands per second) – Consider sharding the key across multiple DO IDs or adding an in‑memory cache inside the DO. Remember the 128 MB memory limit and that each DO still serializes requests for its own ID.

Below is a minimal, safe example of a hybrid pattern that uses the Cache API for hits and a Durable Object for the miss path. It uses the standard Workers event API and avoids unsupported headers.
addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request, event));
});

async function handleRequest(request, event) {
  const cache = caches.default;
  const cached = await cache.match(request);
  if (cached) return cached;

// Miss – delegate to a Durable Object
  const doId = env.MY_DO.idFromName(request.url); // deterministic ID per key
  const doStub = env.MY_DO.get(doId);
  const doResponse = await doStub.fetch(request);

// Store the response in the Cache API with a short TTL
  const headers = new Headers(doResponse.headers);
  headers.set('Cache-Control', 'public, max-age=5, stale-while-revalidate=30');
  const cachedResp = new Response(doResponse.body, {
    status: doResponse.status,
    statusText: doResponse.statusText,
    headers,
  });

// Persist to cache asynchronously
  event.waitUntil(cache.put(request, cachedResp.clone()));

return cachedResp;
}

Diagnostic Question
To decide which strategy is best for your use case, you need one key piece of data:

What is the typical origin response time for the hot key, and how many concurrent requests do you expect during a burst?

Running a load test with wrk or k6 and instrumenting origin request counts will reveal whether the Cache API’s stampede risk outweighs the DO’s serialization cost in your environment.

Cache API vs. Durable Objects for Hot-Key Read Coalescing

Managing Tail Latency Under Concurrent Load

Trade-offs in Coordination

Scheduling Uncertainty

1 answer

Direct Answer

Confirmed Facts (as of 2026‑10‑18)

Uncertain or Unverified Aspects

Practical Guidance

Diagnostic Question

Your perspective belongs here.

0 question comments