gRPC latency spikes when multiple calls share one HTTP/2 connection

gRPC unary RPC latency spikes exclusively when multiple calls share a single HTTP/2 connection

0 reputation · 27 Nov 2021, 16:44 UTC

129.6K views0

gRPC unary RPC latency spikes only when multiple calls run concurrently on a single HTTP/2 connection.

The shared connection limits the HTTP/2 flow‑control window, enforces a max concurrent streams setting, and subjects all streams to TCP head‑of‑line blocking from packet loss.

Uncertainty remains whether the delay stems from application buffer draining, flow‑control window size, or TCP congestion on the shared connection.

Does latency correlate with the server’s max concurrent streams configuration?
Is the spike reproduced when the client drains inbound data promptly versus when it stalls?
Can adjusting the HTTP/2 flow‑control window size mitigate the concurrency‑only latency?

3,924 following

Keep exploring this topic

Raising grpc.max_concurrent_streams above 100: latency spikes under bursty load on a single HTTP/2 connection

Evaluating whether raising grpc.max_concurrent_streams on one HTTP/2 connection fixes burst-only latency, or whether flow-control windows and TCP head-of-line blocking make connection pooling the better trade-off.

gRPC HTTP/2 Flow Control: Tuning Window Sizes for High Concurrency Latency

Analysis of gRPC HTTP/2 flow control window sizes and their impact on latency during high concurrent request loads across different transport implementations.

gRPC max concurrent streams per connection: tuning stream limits versus opening additional connections

Choosing between raising gRPC's max concurrent streams per HTTP/2 connection and opening additional connections, given divergent per-language defaults and interaction with flow control windows.

gRPC-Web Browser Client to Standard gRPC Service: Is a Proxy Required?

Can a browser gRPC-Web client reach a standard gRPC service directly, or is a translation proxy required at the integration boundary?

To isolate whether the latency stems from flow control or TCP behavior, it is important to distinguish between the stream-level and connection-level windows. In HTTP/2, a single slow-consuming stream can exhaust the shared connection window, effectively throttling all other concurrent unary RPCs regardless of their individual health.

Verification Strategies

Packet Loss Simulation: Use a tool like tc (Linux) or Clumsy (Windows) to introduce minimal packet loss. If all multiplexed streams spike simultaneously, the issue is likely TCP head-of-line (HOL) blocking, which is inherent to TCP and cannot be tuned via gRPC settings.
Connection Pooling: Compare the latency of a single shared connection against a pool of multiple connections. If the spikes disappear with a pool, it confirms the bottleneck is the shared HTTP/2 state or TCP congestion.
Window Analysis: Use Wireshark to monitor WINDOW_UPDATE frames. Frequent pauses in data transmission followed by these frames suggest the BDP (Bandwidth Delay Product) is exceeding the current window size.

gRPC unary RPC latency spikes exclusively when multiple calls share a single HTTP/2 connection

0 answers

Your perspective belongs here.

1 question comment

Verification Strategies