HTTP(S) Load Balancing to Cloud Run concurrency cap and tail latency during revision rollout

Answer

Does the load balancer impose additional concurrency limits?

No. The HTTP(S) Load Balancer does not enforce its own per-instance concurrency cap. It terminates connections at the LB level only (via SSL/TLS or HTTP/2 flow control) but forwards requests to Cloud Run instances based on their concurrency setting (default 80). Bursts beyond this limit are handled by Cloud Run's internal queuing or scaling, not the LB.

Confirmed Facts

• Cloud Run instances process up to concurrency requests simultaneously. Excess requests are queued or served by new instances.
• During revision rollouts, the LB performs connection draining (default 10 seconds), waiting for in-flight requests to complete before Cloud Run terminates old instances.
• Tail latency during rollouts typically originates from instance termination timing, not the LB. If an in-flight request exceeds the draining window, it may be terminated mid-flight.
• No documented p99 SLO exists for the LB-to-Cloud Run path. Latency varies by region and revision configuration.

Likely Explanation for Tail Latency

Tail latency during rollouts most often stems from the interaction between:

1. Connection draining timeout (default 10s) expiring before in-flight requests complete.
2. Instance shutdown occurring while concurrent requests are still being processed.
3. Burst traffic triggering new instance startup latency before scaling catches up.

The LB itself does not add latency; it simply waits for Cloud Run to signal readiness to terminate.

Diagnostic Steps

To isolate the source of tail latency:

1. Compare metrics in Cloud Monitoring during steady-state vs rollout periods. Look for spikes in request_latency or instance_startup_time.
2. Adjust connection draining via the Cloud Run service settings if requests routinely exceed 10 seconds.
3. Test with synthetic bursts to observe queuing behavior and verify that latency scales with instance startup time, not LB behavior.

Recommendation

If tail latency persists during rollouts, increase the connection draining timeout or reduce request duration. The LB is not the bottleneck; Cloud Run's instance lifecycle management is.