Configuring gRPC Keepalive to Prevent Silent Connection Drops

If a gRPC channel sits idle for several minutes, the first RPC that follows often hangs or fails with UNAVAILABLE. The transport was silently dropped by a NAT gateway, firewall, or load balancer that expired its flow-table entry. gRPC keepalive prevents this by probing idle connections with HTTP/2 PING frames. The practical decision is to enable keepalive on both the client and the server, with a ping interval shorter than the shortest intermediary idle timeout in your network path, but longer than any minimum client ping rate the server enforces.

How HTTP/2 keepalive detects a dead path

HTTP/2 defines a PING frame that carries an opaque payload. The receiver must reply with a PING ACK containing the same payload. gRPC uses this as a liveness probe: if a side sends a PING and does not receive the matching ACK within a configured timeout, it closes the transport. The channel then reconnects on the next RPC instead of blocking on a dead socket.

Both directions matter. A client-side keepalive detects when the server or an intermediary has disappeared. A server-side keepalive detects when a client has vanished without closing the TCP connection. Because abusive clients could otherwise flood a server with PING frames, gRPC servers also enforce a minimum client ping rate. If a client pings more often than allowed, the server terminates the transport with a GOAWAY frame.

A worked server and client configuration

The following example uses the Go gRPC library. Parameter names and defaults differ across language implementations and versions; consult the channel or server builder for your runtime rather than assuming cross-language parity.

Server side

Run this in your server initialization code. The server sends a PING every 30 seconds and waits 10 seconds for the ACK. The enforcement policy rejects clients that ping more often than every 5 seconds, while still permitting pings when no RPCs are active.

import (
    "time"

    "google.golang.org/grpc"
    "google.golang.org/grpc/keepalive"
)

s := grpc.NewServer(
    grpc.KeepaliveParams(keepalive.ServerParameters{
        Time:    30 * time.Second, // ping interval
        Timeout: 10 * time.Second, // wait for ACK
    }),
    grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
        MinTime:             5 * time.Second,
        PermitWithoutStream: true,
    }),
)

PermitWithoutStream: true on the enforcement policy allows clients to probe the connection even when no RPCs are in flight. If your intermediaries drop idle connections aggressively, leave this enabled; if you want to restrict idle probing entirely, set it to false.

Client side

Run this where you create the client channel. The values mirror the server, but the client also decides whether to send pings without active streams.

conn, err := grpc.Dial("grpc.example.internal:50051",
    // credentials omitted for brevity
    grpc.WithKeepaliveParams(keepalive.ClientParameters{
        Time:                30 * time.Second,
        Timeout:             10 * time.Second,
        PermitWithoutStream: true,
    }),
)

Set PermitWithoutStream to true when your load balancer or NAT has an idle timeout shorter than your longest expected gap between RPCs. Set it to false if your proxy terminates connections that carry PING frames without an active stream, or if you want to minimize background traffic.

Limits: what keepalive cannot fix

Keepalive detects a dead transport, not a hung application process. If the server process is alive but its event loop is blocked, the HTTP/2 layer may still answer PING frames while RPCs stall. Keepalive also cannot bypass middleboxes that buffer or silently discard HTTP/2 PING frames; in those environments the probe never reaches the peer. Finally, overly aggressive settings cause connection churn: pings that arrive faster than the server’s MinTime trigger GOAWAY, and intervals that are too short waste bandwidth and CPU.

Common mistakes and how to avoid them

• Enabling keepalive on only one side. A client-only configuration detects server-side drops but not clients that vanish without closing the connection. A server-only configuration protects the server but leaves clients blind to NAT expiry. Configure both.
• Choosing an interval shorter than the intermediary idle timeout without accounting for ping-rate enforcement. If your NAT expires flows after 60 seconds but your server enforces a 10-second minimum client ping interval, a 5-second client ping will be rejected with GOAWAY. Align the client interval, the server enforcement policy, and the network idle timeout.
• Relying on keepalive alone. Keepalive closes a dead transport; it does not retry the RPC. Implement retries with backoff and connection draining so that in-flight requests survive a GOAWAY or a forced reconnection.

Verifying the behavior in your network

Capture traffic on the client host to confirm that PING and PING ACK frames appear at the configured interval during idle periods. Run the following with root or CAP_NET_RAW privileges, replacing <interface> and <port> with your values:

sudo tcpdump -i <interface> -w grpc-keepalive.pcap tcp port <port>

Open the capture in Wireshark and filter for HTTP/2 PING frames (frame type 6). You should see a PING followed by a PING ACK roughly every 30 seconds. If you see GOAWAY frames instead, the server is rejecting your ping rate; increase the client interval or relax MinTime. To test the dead-path scenario, place a stateful firewall or NAT between client and server with a short idle timeout and verify that the connection survives or reconnects cleanly without RPC failures.