Using gRPC Client‑Side Load Balancing Without an External Proxy

Problem: spreading RPCs across instances without a sidecar

When you run multiple identical gRPC servers behind a DNS name, each client call currently hits the first resolved address. If you want the client to distribute traffic evenly—so you can add or remove instances without re‑configuring a central load balancer—you need a way for the client itself to pick a subchannel for each RPC.

Thesis: enable client‑side load balancing with a service config

gRPC lets you attach a service configuration JSON to a client channel. The config names a load‑balancing policy (e.g., round_robin) that works with the resolver’s address list. The client then invokes the policy for every outgoing RPC, selecting a subchannel without any external proxy.

When to use it

• You control the client stub and can inject a service config.
• The service instances are homogeneous and reachable via a single DNS name.
• Instances are added or removed frequently, making static client lists impractical.
• You prefer to keep the data plane simple and avoid an extra hop through a sidecar or external load balancer.

Worked example: round‑robin in Go

Assume you have two identical gRPC servers running on ports 50051 and 50052. The client will be configured with a service config that tells gRPC to use round_robin.

1. Prepare the service config JSON

{
  "loadBalancingPolicy": "round_robin",
  "healthCheckConfig": {
    "serviceName": "myservice"
  }
}

Save this as lb_config.json. The healthCheckConfig block is optional but shows where you could add health‑checking later.

2. Load the config when creating the client

import (
    "context"
    "log"
    "time"

    "google.golang.org/grpc"
    "google.golang.org/grpc/balancer/roundrobin"
    "google.golang.org/grpc/resolver"
    "google.golang.org/grpc/serviceconfig"
)

func main() {
    // Register the round_robin balancer so the resolver can find it.
    balancer.Register(roundrobin.NewBuilder())

    data, err := os.ReadFile("lb_config.json")
    if err != nil {
        log.Fatalf("failed to read lb config: %v", err)
    }
    var cfg serviceconfig.ParseResult
    if err := json.Unmarshal(data, &cfg); err != nil {
        log.Fatalf("failed to parse service config: %v", err)
    }

    // Build dial options that inject the parsed config.
    opts := []grpc.DialOption{
        grpc.WithDefaultServiceConfig(&cfg), // attaches the config
        grpc.WithBlock(),
        grpc.WithInsecure(), // for demo only; use TLS in prod
    }

    conn, err := grpc.DialContext(context.Background(), "dns:///myservice.example.com", opts...)
    if err != nil {
        log.Fatalf("failed to dial: %v", err)
    }
    defer conn.Close()

    client := pb.NewMyServiceClient(conn)
    // Example RPC loop
    for i := 0; i < 20; i++ {
        resp, err := client.Echo(context.Background(), &pb.EchoRequest{Msg: "hello"})
        if err != nil {
            log.Printf("rpc %d failed: %v", i, err)
            continue
        }
        log.Printf("rpc %d reply: %s", i, resp.Msg)
        time.Sleep(100 * time.Millisecond)
    }
}

Explanation:

• The resolver (here DNS) returns a list of addresses for myservice.example.com.
• The round_robin balancer picks the next subchannel for each RPC.
• The service config is passed via WithDefaultServiceConfig, which tells gRPC to use the policy.

3. Run the servers

In two separate terminals:

# Terminal 1
 go run server.go -port 50051

# Terminal 2
 go run server.go -port 50052

Make sure the servers register the same service name under myservice.example.com in your DNS or /etc/hosts file.

Trade‑off and limitation

The main cost is added client‑side complexity:

• You must ensure the resolver returns fresh addresses; a high DNS TTL can cause the client to keep using dead instances until the next resolution cycle.
• The client does not automatically retry a failed RPC on another subchannel; you need to implement retry logic at the application level if you want that behavior.
• Latency spikes can appear if the chosen subchannel is slow or unhealthy, because the client sticks with it until the balancer’s next pick (which for round_robin is the next RPC).
• You still need to configure mutual TLS, tracing, and metrics separately; client‑side load balancing does not replace those concerns.

How to verify the balancing works

Enable verbose logging on the client (set environment variable GRPC_GO_LOG_VERBOSITY_LEVEL=2 or GRPC_TRACE=all) and look for lines like:

grpc: picked subchannel index 0
grpc: picked subchannel index 1

Over a short burst of RPCs you should see roughly equal counts for each index.

Alternatively, watch the server logs: each server should report handling about half of the requests. You can also inspect the channel state via the gRPC Go zpage (http://localhost:8080/debug/grpc) if you enabled the grpc_zstats handler, which shows the picker being invoked per RPC.

Actionable closing

If your service‑to‑service traffic is homogeneous and you can adjust the client stub, start with the built‑in round_robin policy. Test with two instances, verify the picker alternates, and monitor for stale DNS or latency outliers. When you need more sophisticated algorithms (e.g., least‑request) or want to centralize policy updates, consider migrating to the grpc_lb balancer that talks to an external load‑balancing service, but keep in mind the same operational overhead of maintaining that external service.