Application-Level Flow Control for gRPC Bidirectional Streams

The symptom: memory grows when one consumer slows down

You expose a live feed through a gRPC bidirectional stream. The server pushes events; the client receives them. Under a spike, a client on a spotty network stops reading for a few seconds. Because the server keeps writing into the stream, its in-memory send queue grows. HTTP/2 flow control eventually pauses the TCP bytes, but your application has already allocated the messages. The container hits its memory limit and restarts.

This is the gap between transport flow control and application backpressure. HTTP/2 WINDOW_UPDATE frames tell the sender when the network buffer has room. They do not know how many Protobuf messages your service has queued in application memory.

Design: credit-based flow control on the stream

Add an explicit credit field to the client-to-server direction of the stream. The client promises it can accept N more messages. The server sends only while credits remain, then pauses until the client sends another grant.

syntax = "proto3";

package feed.v1;

service EventFeed {
  rpc Subscribe(stream SubscribeRequest) returns (stream Event);
}

message SubscribeRequest {
  string cursor = 1;
  int64 grant = 2;
}

message Event {
  string id = 1;
  bytes payload = 2;
}

The server loop looks conceptually like this:

for {
  select {
  case <-ctx.Done():
    return ctx.Err()
  case <-credits:
    if err := stream.Send(nextEvent()); err != nil {
      return err
    }
  }
}

If credits is empty, the goroutine blocks instead of allocating more events. When the client sends a new SubscribeRequest with a larger grant, the server resumes. This is application-level backpressure: the producer cannot run ahead of the consumer by more than the outstanding credit.

Lifecycle, channels, and deadlines

• Half-close: the client should end its request stream when it is done sending subscriptions or cursors. The server must not wait forever for a request that will never arrive.
• Cancellation: watch the RPC context. A client disconnect, deadline, or manual cancel ends the entire stream, not just one message.
• Deadlines: a 30-second deadline cancels a 10-minute feed. For long sessions, send heartbeat events or refresh the deadline from the client before it expires.
• Channel separation: use a dedicated gRPC channel for streaming traffic. Long-lived streams occupy HTTP/2 connections and can block unary calls behind them. Check your client's maxConcurrentStreams limit; a common default is 100 per connection.
• Load balancing: a single stream stays on one backend. Do not expect per-message load balancing across a stream; create a new stream per client session and let the load balancer pick the backend at stream open.

Worked check: reproduce the slow-consumer problem locally

Run this in a local development environment, not production. You need only a running gRPC server and client on the same machine; no special permissions are required.

1. Start the EventFeed.Subscribe server with credit-based sending disabled, so it writes as fast as it can.
2. Connect a client that reads exactly one Event, then stops reading from the stream.
3. Watch the server process memory. If it climbs steadily while the client is silent, the server is buffering without backpressure.
4. Enable credit-based sending, set an initial grant of one, and repeat. The server should pause after the first message and resume only when the client sends another grant.

Expected check: memory stays flat and the server logs a “credit exhausted” event. Risk: if the client never sends another credit, the stream stalls permanently. Add a stream-level timeout or a client liveness check so a dead client does not hold server resources forever.

Trade-offs and limitations

Concern	Behavior
Head-of-line blocking	One slow consumer blocks its own stream, not other streams on the same connection, but the connection is still shared.
Session resumption	gRPC has no built-in resume after a network change. Use cursor-based resumption or sequence numbers in the application layer.
Schema evolution	Adding or removing streamed message types breaks compatibility. Version the package, for example feed.v1 and feed.v2.
Wrong tool for CRUD	Use unary RPCs for create, read, update, delete. Bidirectional streaming adds latency and complexity for request/response workloads.

What to do next

Pick one streaming method, add a credit field to the client request, and route streaming traffic through its own channel. Instrument stream open, close, messages sent, messages received, and “credit exhausted” events. Alert when a stream stays stalled longer than your SLO. If memory stays flat during a slow-consumer test, your backpressure is working.