Cache API vs. Durable Objects for Hot-Key Read Coalescing

Managing Tail Latency Under Concurrent Load

When implementing a high-traffic read endpoint in Cloudflare Workers, managing latency spikes during concurrent bursts is critical. Two documented approaches exist for handling "hot keys" to prevent origin overload and reduce tail latency: the Cache API and Durable Objects.

Trade-offs in Coordination

The Cache API allows for fast edge reads and can absorb repeated identical requests at the colo. However, it does not provide strong cross-request serialization, which may lead to cache stampedes where multiple concurrent requests trigger duplicate origin fetches during a miss or eviction.

Conversely, Durable Objects provide a single-threaded execution environment for a specific ID, allowing the developer to collapse duplicate in-flight operations into a single request. While this prevents stampedes, it introduces an additional network hop and creates a potential bottleneck if the QPS for a single object exceeds the isolate's processing capacity.

Scheduling Uncertainty

Because JavaScript execution within a single isolate is single-threaded, CPU-bound work can delay other requests scheduled on that same isolate. It remains unclear how Cloudflare's scheduler distributes bursts of concurrent requests across isolates for a single worker script.

• Does the Cache API's lack of serialization consistently result in higher origin load compared to the overhead of a Durable Object hop during a burst?
• Under high concurrency, does the single-threaded nature of a Durable Object introduce more tail latency than the potential for duplicate fetches in the Cache API?