pick_first vs round_robin: Trading Failover Predictability for Load Distribution in Polyglot gRPC Services

Direct answer

Yes. Standardizing on pick_first and moving load distribution to an Envoy sidecar is the more predictable option for cross-language failover. pick_first pins a client to one endpoint until that endpoint fails, so failover is triggered by connection errors rather than by each runtime's DNS refresh cadence. An Envoy sidecar centralizes DNS resolution and load-balancing decisions, so Java, Go, and Python clients share one TTL and one unhealthy-endpoint removal path.

Likely explanation versus established behavior

Established gRPC behavior: pick_first selects a single address and switches only on explicit failure; round_robin distributes new calls across resolved addresses. The gRPC specification does not mandate name-resolver re-resolution intervals. Java's DnsNameResolver often uses a fixed cache refresh interval, Go's resolver typically follows DNS TTL with caching, and Python's resolver generally follows the OS DNS cache. Those differences are runtime- and version-specific, so round_robin failover timing naturally diverges across languages.

Because the specification leaves resolver timing to each implementation, standardizing on round_robin alone cannot guarantee uniform failover. A sidecar collapses that variance to the sidecar's own resolution interval, which is easier to operate consistently than tuning three client libraries.

Steps for this case

1. Configure Java, Go, and Python gRPC clients to use pick_first against the sidecar's local listener.
2. Run an Envoy sidecar per client host or per client process.
3. Configure the Envoy cluster with DNS resolution and health checking matched to your failover SLO.
4. Set a short, uniform TTL on the backend DNS record and remove per-client round_robin policy overrides.

If round_robin is required

Force all clients to the same resolution source. Options, from simplest to most invasive:

• Publish a static DNS TTL of five seconds or less and shorten or disable OS DNS caching on client hosts.
• Use a custom gRPC NameResolver in Java and Go that queries a service registry directly, bypassing OS cache.
• Keep round_robin in clients but point them at the sidecar, letting Envoy perform distribution.

None of these guarantees under-five-second variance unless you confirm that no intermediate resolver, container platform, or mesh control plane adds its own cache. Treat the under-five-second target as unverified until measured.

Missing diagnostic detail

Are the clients already running behind sidecar proxies, or would Envoy be new infrastructure? If Envoy is new, the operational cost of running and observing sidecars may outweigh the predictability benefit; fixing DNS TTL and resolver settings may be the more practical first step.

Verification

Stop one backend endpoint while Java and Go clients send steady traffic, then measure the time from failure to the first successful request on the remaining endpoint. Repeat with identical DNS TTL settings and compare against your five-second target.

# Read-only inspection of JVM DNS cache TTL
jinfo -flag sun.net.client.defaultAddressCacheTtl <pid>

# Envoy cluster endpoint status (read-only)
curl -s http://localhost:9901/clusters?format=json

Assumptions: clients use default or passthrough name resolution; Envoy is co-located with clients; backends are addressable by DNS. Uncertainty: exact failover variance under load is environment-specific and requires measurement.