Latency spikes under concurrent bursts in Cosmos DB autoscale containers

Question

An application using Cosmos DB autoscale provisioned throughput reports elevated request latency that appears only when many clients issue requests simultaneously. Single-client latency remains normal, and RU consumption metrics show a burst that precedes the latency increase. The concern is whether the delay between detecting increased demand and the actual RU/s scale-up is the primary cause, or whether partition hot-spotting and client connection pool exhaustion are contributing factors. Public documentation does not specify the exact timing, magnitude, or regional consistency of the autoscale ramp-up behavior, making it difficult to predict whether latency will recover before requests time out. The interaction between autoscale and multi-region writes or change feed processing under concurrent load is also unclear.

Is the latency spike primarily a consequence of the autoscale RU/s ramp-up delay, or should partition distribution and client-side connection limits be ruled out first? Does the scale-up timing and resulting latency vary by region or SDK version in ways that current documentation does not capture? What observable metrics or diagnostic signals can distinguish autoscale ramp-up latency from partition hot-spotting under concurrent bursts?

Tasadduq Burney · Answer

Answer
Likely cause
Latency spikes are most often a result of the autoscale service needing a few seconds to increase provisioned RU/s after a burst of requests. During that window the request charge stays high, causing throttling (429) and higher response times until the new RU/s level is provisioned.
Confirmed facts

Autoscale adds RU/s within a typical 5‑second window after sustained high request unit usage.
If a burst exceeds the current RU/s before this window finishes, requests experience throttling and latency until the scale‑up completes.
Partition hot‑spotting creates uneven RU/s consumption on a few logical partitions, which can also lead to latency even when overall autoscale capacity is sufficient.
Client‑side connection pool limits (e.g., default max connections per host in SDKs) can queue requests, adding latency without generating 429 responses.

Verification steps

Enable Azure Monitor diagnostics for the Cosmos DB account and capture metrics: Autoscale Max RU/s, Autoscale Current RU/s, Request Charge, and HTTP 429 counts.
Run a controlled burst test that generates a sharp increase in request rate while monitoring the above metrics.
Correlate latency spikes with periods where Request Charge approaches or exceeds Autoscale Current RU/s and 429 responses rise before Autoscale Max RU/s increases.
Examine per‑partition RU/s usage (available via diagnostic logs or the analytical store) to detect uneven distribution that aligns with latency spikes.
Check client SDK connection‑pool metrics (e.g., HttpClientPool size, async connection pool saturation) for exhaustion signs concurrent with latency increases.
If latency persists after autoscale has scaled up and no 429 errors are observed, investigate partition hot‑spotting or client‑side limits as remaining causes.

Missing diagnostic detail
Confirm whether per‑partition RU/s usage or client connection‑pool saturation metrics are available during the burst; this information will determine if hot‑spotting or client limits are the dominant factor.

Latency spikes under concurrent bursts in Cosmos DB autoscale containers

1 answer

Answer

Likely cause

Confirmed facts

Verification steps

Missing diagnostic detail

Your perspective belongs here.

0 question comments