gRPC Server Reflection in production: alerting on usage without notification noise
0 reputation · 24 Mar 2022, 08:05 UTC
0 reputation · 24 Mar 2022, 08:05 UTC
I run gRPC services (Go and Java servers, assume grpc-go v1.6x and grpc-java 1.6x) with Server Reflection enabled so internal tooling like grpcurl and our schema registry can discover services without shipping proto files. I want alerts that tell me when reflection is being used in a way that matters, without paging on every legitimate lookup.
Reflection has no built-in permission model in the core gRPC spec, so any restriction is implemented by me: network policy, interceptor-based auth checks, or filtering which services are advertised. That means my alerting has to live outside the framework too, likely as metrics from a server interceptor counting reflection requests by peer identity and requested symbol.
The hard part is defining the signal. CI pipelines and developer laptops generate bursty, legitimate reflection traffic, so a simple request-rate threshold will either page constantly or miss slow reconnaissance. I also cannot tell from documentation alone whether per-service filtering of advertised symbols is uniformly supported across the two runtimes I use, or whether behavior differs by version.
A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.