Choosing a gRPC Streaming Mode for Telemetry Ingestion

The decision and constraints

You are building a telemetry ingestion service. Edge gateways and mobile clients send small, frequent sensor batches. The service must acknowledge each batch, push threshold alerts without polling, apply backpressure to slow consumers, and tolerate variable latency on unreliable networks. Clients are heterogeneous, including browsers reached through gRPC-Web.

Useful takeaway: choose client-streaming for ingestion batches and server-streaming for alert subscriptions. Reserve bidirectional streaming for cases where client and server must interleave messages in real time, and avoid unary for high-frequency batch uploads.

Supported streaming modes compared

Mode	Client sends	Server sends	Fits telemetry	Main risk
Unary	One message	One message	Occasional configuration fetch	Head-of-line blocking; one connection per logical exchange
Server-streaming	One message	Many messages	Push alerts or threshold breaches	Slow client can stall the stream; proxies may buffer responses
Client-streaming	Many messages	One message	Sensor batch upload from gateways	Partial failures; server may commit work before returning an error
Bidirectional	Many messages	Many messages	Real-time sync or chat-like telemetry	Complex flow control; hung peers can leak server resources

Trade-offs to weigh

Unary

Unary is the simplest contract: one request, one response. For telemetry it forces the client to open a new exchange for every reading or batch, which adds latency and connection overhead under load. It also hides per-message latency because the client waits for the full response before sending again.

Server-streaming

Server-streaming suits alert delivery. The client sends a filter once and receives a stream of alerts. HTTP/2 flow control operates per-stream and per-connection, so the server must respect the client's read window; otherwise a slow consumer stalls the connection. gRPC-Web proxies in older environments may buffer streaming responses, effectively turning server-streaming into unary.

Client-streaming

Client-streaming lets an edge gateway pipeline many SensorReading messages and half-close when the batch is complete. The server reads from the request iterator, aggregates, and returns a single IngestSummary. The risk is error semantics: if the server commits side effects while reading, it may have already acted on messages when it finally returns an error.

Bidirectional streaming

Bidirectional streaming maximizes throughput and minimizes latency, but both sides must manage explicit backpressure and deadline propagation. Without deadlines, a hung peer can hold server resources indefinitely. Use it only when ingestion and alerts must interleave on the same stream.

Recommended proto design

syntax = "proto3";

package telemetry;

service IngestService {
  // Client-streaming: gateway batches readings, server returns one summary.
  rpc IngestBatch(stream SensorReading) returns (IngestSummary);

  // Server-streaming: client subscribes to alerts without polling.
  rpc SubscribeAlerts(AlertFilter) returns (stream Alert);
}

message SensorReading {
  string device_id = 1;
  double value = 2;
  int64 timestamp_ms = 3;
}

message IngestSummary {
  int64 accepted_count = 1;
  double total_value = 2;
}

message AlertFilter {
  string severity = 1;
}

message Alert {
  string device_id = 1;
  string message = 2;
}

Version assumption: this proto uses proto3 and is compatible with gRPC implementations that support streaming RPCs, including grpc-go, grpc-java, and grpc-python.

Server handler sketch

The following Go-like example shows the ingestion handler. It reads until the client half-closes, aggregates readings, and returns one summary. It checks the context deadline on every iteration so a slow or disappeared client does not leak a goroutine.

func (s *server) IngestBatch(stream grpc.ServerStream) error {
    var summary IngestSummary

    for {
        reading, err := stream.Recv()
        if err == io.EOF {
            // Client half-closed; finalize the batch.
            return stream.SendAndClose(&summary)
        }
        if err != nil {
            return err
        }

        // Application-level aggregation. Commit only after the batch
        // completes if you need all-or-nothing semantics.
        summary.AcceptedCount++
        summary.TotalValue += reading.Value

        if err := stream.Context().Err(); err != nil {
            return err
        }
    }
}

Read promptly from the client stream. If the server pauses while the client keeps writing, HTTP/2 flow control can stall the entire connection, not just this stream.

Validation with grpcurl and a small client script

Run these checks from a machine that can reach the service. No special permissions are required beyond network access.

1. Confirm the service is registered:

grpcurl -plaintext <host>:<port> list

Expected: the list includes telemetry.IngestService. Risk: a missing entry means the server did not bind the service or is listening on a different port.

1. Inspect the streaming signatures before writing client code:

grpcurl -plaintext <host>:<port> describe telemetry.IngestService.IngestBatch
grpcurl -plaintext <host>:<port> describe telemetry.IngestService.SubscribeAlerts

Expected: IngestBatch shows a streaming request and unary response; SubscribeAlerts shows a unary request and streaming response.

1. Send a unary probe to verify the endpoint is reachable:

grpcurl -plaintext -d '{"severity":"high"}' <host>:<port> telemetry.IngestService/SubscribeAlerts

Expected: the server begins streaming alerts; terminate with Ctrl-C. If an intermediary buffers the stream, you may receive one large response instead of incremental messages.

1. Verify client-streaming aggregation with a small script:

• Open a client stream to IngestBatch.
• Send N SensorReading messages.
• Half-close the stream (for example, CloseSend in grpc-go).
• Read the single IngestSummary response and assert AcceptedCount == N.

Risk: if the client does not half-close, the server blocks waiting for more messages and never returns the summary.

Flow-control check

To verify backpressure is active, slow the server's read loop artificially and observe the client. The client's writes should block once the HTTP/2 connection window is exhausted. You can confirm with tcpdump or a local proxy that WINDOW_UPDATE frames flow only after the consumer reads data. If frames keep flowing while the server is stalled, an intermediary is buffering and defeating flow control.

Limitations

• gRPC-Web streaming support varies by browser and proxy; older proxies may buffer server-streaming responses.
• Client-streaming errors can arrive after partial side effects. Design ingestion to be idempotent or commit only after the stream ends.
• Bidirectional streams require explicit deadlines; without them, a hung peer leaks server resources.
• This guide does not cover authentication, TLS, or load balancing, which are required for production telemetry endpoints.

How to check the result

• Run grpcurl list and describe to confirm the expected streaming methods exist.
• Run the client-streaming script with N messages and confirm the server returns AcceptedCount == N after half-close.
• Use a proxy or tcpdump to confirm HTTP/2 DATA frames move in both directions and are not buffered by an intermediary.

No runtime rollback is required because this guide selects an API design and validation steps; proto changes are file-level edits, not live state mutations.