Building Low‑Latency Real‑Time Services with gRPC Bidirectional Streaming

Problem: needing instant, two‑way communication without the cost of many HTTP connections

When you design a service that must push updates to many clients while also receiving commands from them (think live chat, collaborative editing, or telemetry dashboards), opening a separate unary RPC for each message adds latency and consumes extra HTTP/2 frames. The naive approach of polling or opening many streams can quickly exhaust server resources and make the system hard to reason about.

Thesis: gRPC bidirectional streaming gives a single, multiplexed channel that lets client and server exchange messages independently, reducing round‑trip latency and simplifying the event‑driven model.

Core concept

In gRPC, a bidirectional stream is defined in a .proto file as a service method where both the request and response types are streams:

service ChatService {
  rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}
message ChatMessage {
  string user_id = 1;
  string text    = 2;
}

When a client calls Chat, gRPC upgrades the underlying HTTP/2 connection to a single bidirectional stream. Both sides can write and read messages at any time, and the HTTP/2 layer multiplexes frames so that one slow message does not block another (no head‑of‑line blocking).

Worked example: a minimal chat service in Go

Below is a concise implementation that shows the essential steps. You can copy the snippets into two files, server.go and client.go, and run them locally.

Server

package main

import (
	"context"
	"log"
	"net"

	"google.golang.org/grpc"
	pb "example.com/chatpb"
)

type server struct {
	pb.UnimplementedChatServiceServer
	streams map[chan *pb.ChatMessage]struct{}
}

func (s *server) Chat(stream pb.ChatService_ChatServer) error {
	msgChan := make(chan *pb.ChatMessage, 64)
	s.streams[msgChan] = struct{}{}
	delete(s.streams, msgChan) // cleanup on return
	defer close(msgChan)

	// Goroutine to broadcast incoming messages to all other streams
	go func() {
		for {
			in, err := stream.Recv()
			if err != nil {
				log.Printf("receive error: %v", err)
				return
			}
			for ch := range s.streams {
				select {
				case ch <- in:
				default:
				}
			}
		}
	}()

	// Send messages received on msgChan back to this client
	for msg := range msgChan {
		if err := stream.Send(msg); err != nil {
			return err
		}
	}
	return nil
}

func main() {
	lis, err := net.Listen("tcp", ":50051")
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterChatServiceServer(s, &server{streams: make(map[chan *pb.ChatMessage]struct{})})
	log.Println("server listening on :50051")
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

Where to run: In a terminal with Go 1.22+ installed, execute go run server.go. No special privileges are needed unless you choose a port < 1024.

Client

package main

import (
	"context"
	"log"
	"time"

	"google.golang.org/grpc"
	pb "example.com/chatpb"
)

func main() {
	conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure(), grpc.WithBlock())
	if err != nil {
		log.Fatalf("did not connect: %v", err)
	}
	defer conn.Close()
	client := pb.NewChatServiceClient(conn)

	stream, err := client.Chat(context.Background())
	if err != nil {
		log.Fatalf("open stream failed: %v", err)
	}

	// Send messages from stdin
	go func() {
		var text string
		for {
			if _, err := fmt.Scanln(&text); err != nil {
				continue
			}
			if err := stream.Send(&pb.ChatMessage{UserId: "alice", Text: text}); err != nil {
				log.Printf("send error: %v", err)
				return
			}
		}
	}()

	// Receive and print messages
	for {
		msg, err := stream.Recv()
		if err != nil {
			log.Printf("receive error: %v", err)
			break
		}
		log.Printf("%s: %s", msg.GetUserId(), msg.GetText())
	}
}

Where to run: In another terminal, execute go run client.go. The client will block waiting for input; type a line and press Enter to send a message. You should see the message appear instantly in the other client’s console.

Trade‑offs and limitations

• Stream lifecycle complexity: Unlike unary RPCs, you must explicitly close streams, handle client disconnects, and clean up server‑side resources. Forgetting to delete a stream from a broadcast list can cause memory leaks.
• Back‑pressure: If a client reads slower than the server writes, the server’s send buffer may fill. The Go implementation will block on Send until space is available, which can stall other goroutines unless you use buffered channels or context cancellation.
• Debugging difficulty: Log lines from many streams interleave, making it hard to trace a single conversation. Adding a stream identifier (e.g., the peer address or a UUID) to each log entry and enabling gRPC’s tracing (grpc-go stats handler) mitigates this.
• Resource exhaustion: Idle clients that keep a stream open consume a file descriptor and a goroutine on the server. Implement an idle timeout (e.g., close streams with no activity for 30 s) and enforce a maximum concurrent stream count.

Verification steps you can perform

1. Functional test: Run the server and two clients as described. Send a message from client A; verify that it appears in client B’s output within a few milliseconds (you can measure with time or a simple timestamp in the message).
2. Metrics check: In the server, attach the grpc-go stats handler (stats.Handler) to collect msg_sent and msg_received per stream. After a burst of 100 messages, confirm that the counters increase accordingly.
3. Network inspection: With Wireshark or tcpdump -i lo port 50051 -w capture.pcap, examine the trace. You should see a single HTTP/2 connection per client containing both HEADERS and DATA frames flowing in both directions, confirming multiplexing.

Actionable closing

If your application requires low‑latency, bidirectional data flow and you can tolerate the added responsibility of managing stream lifecycles, gRPC bidirectional streaming is a solid choice. Start by defining a clear proto contract, implement a small proof‑of‑concept like the chat service above, and instrument your server with idle‑timeouts and metrics before moving to production load.