Idle Connection Closed Prematurely in gRPC Java Client Due to Platform‑Dependent Default Timeout
0 reputation · 02 Jun 2022, 09:46 UTC
0 reputation · 02 Jun 2022, 09:46 UTC
When using the gRPC Java Netty transport, long‑running services occasionally observe that idle connections are closed after a short period of inactivity, even though no explicit idle timeout has been configured.
The library selects a platform‑dependent default value for the Netty idle‑timeout handler, and the exact number is not documented in the API reference. This makes it difficult to predict whether a connection will survive hours or days without traffic, especially when keepalive is disabled.
What is the precise default idle‑timeout value applied on Linux, Windows, and macOS? How does this default interact with Netty’s IdleStateHandler and any server‑side keepalive settings? Should applications explicitly set a custom idle timeout via ManagedChannelBuilder.idleTimeout, and if so, what value is considered safe for services that expect multi‑hour idle periods?
The idle connection is not being closed by a hidden, platform-dependent Netty default. In gRPC Java, ManagedChannelBuilder.idleTimeout has an explicit, documented default of 30 minutes, and it is the same on Linux, Windows, and macOS. Nothing in the channel-idle path reads an OS setting. If your connections die after a short idle period, the 30-minute channel idle timer — or something else entirely, like a load balancer — is the cause, not a per-OS constant.
The idle timeout is a channel state timer, not a TCP socket timeout. When a channel has had no in-flight RPCs for the configured duration, gRPC transitions the channel to the IDLE connectivity state and tears down the underlying HTTP/2 connection to free resources. The next RPC transparently reconnects. So the closure is by design, and the "premature" feeling usually comes from one of two things:
These mechanisms are independent, and conflating them is the usual misconfiguration:
idleTimeout — closes the channel after inactivity. Default 30 minutes.keepAliveTime — sends HTTP/2 PINGs on an otherwise idle connection to keep it (and middlebox state) alive. Disabled by default on the client.permitKeepAliveWithoutCalls and permitKeepAliveTime — govern whether the server tolerates client pings when no RPCs are active. Aggressive client keepalive without server permission gets connections GOAWAY'd.IdleStateHandler is an internal detail of the transport pipeline; application code should not tune it directly. gRPC's own settings sit above it.OS TCP keepalive (e.g., Linux's 7200-second default) is yet another layer and only matters if gRPC's HTTP/2 keepalive is off — and 2 hours is far too slow to rescue a connection through a stateful middlebox anyway.
Yes, set the timeout explicitly rather than relying on the default:
ManagedChannelBuilder.forAddress(host, port)
.idleTimeout(8, TimeUnit.HOURS)
.keepAliveTime(5, TimeUnit.MINUTES)
.keepAliveTimeout(20, TimeUnit.SECONDS)
.keepAliveWithoutCalls(true);
Guidance:
idleTimeout to a value comfortably above your longest expected quiet period, or keep the 30-minute default if reconnection cost is acceptable — reconnecting is cheap and often the healthiest option.keepAliveTime shorter than your shortest middlebox timeout (5 minutes is a common safe floor; 60 seconds if you traverse aggressive LBs), or the connection will be silently dead when you return to it.permitKeepAliveWithoutCalls(true) and a permitKeepAliveTime at or below the client's ping interval.Before changing values, confirm which timer is firing. Poll channel.getState(true) during the idle window: a transition to IDLE at ~30 minutes confirms the channel timer; a channel stuck in READY whose next RPC fails with UNAVAILABLE points to a middlebox kill. Enabling FINE logging for io.grpc.netty will show the connection teardown events with timestamps. Note that exact defaults have been stable across recent 1.x releases, but verify against the ManagedChannelBuilder Javadoc for your pinned version if you depend on the 30-minute figure.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.