Does Envoy trigger 503 errors when the rate limit service returns an empty allow list?

Question

When implementing a rate-limiting strategy in Envoy (v1.25+), the rate_limit filter communicates with an external gRPC service to determine if a request should be permitted. To facilitate integration testing without production credentials, a mock service can be used to simulate these decisions.

A specific concern arises during high QPS scenarios when the rate limit service responds with a 200 OK status but provides an empty allow list. There is uncertainty regarding whether Envoy interprets this empty response as a successful "allow" or if it triggers a failure state.

If the service returns an empty list under heavy load, does Envoy default to passing the traffic, or does it result in intermittent 503 Service Unavailable errors? How does this behavior differ from a scenario where the service is completely unreachable?

Tasadduq Burney · Answer

Answer
No, Envoy does not trigger a 503 error when the rate limit service returns a 200 OK status with an empty allow list. Instead, Envoy interprets an empty allowed_quota list as a denial of the request, which by default results in a 429 Too Many Requests response to the client.

Likely Explanation of Behavior
In Envoy (v1.25+), the rate_limit filter distinguishes between a service failure and a policy denial. When the external gRPC service is reachable and responds successfully, the decision is based on the content of the response:

Successful Allow: The service returns a non-empty list of quotas. Envoy passes the traffic.
Policy Denial: The service returns a 200 OK but the allowed_quota list is empty. Envoy treats this as "no quota available" and returns a 429.
Service Failure: The service is unreachable, times out, or returns a gRPC error status. This is when Envoy triggers a 503 Service Unavailable (unless configured otherwise via failure policies).

Verification Steps
To confirm this behavior in your specific environment, you can use the following scoped tests:

Test Policy Denial: Configure your mock service to return a gRPC OK status with an empty allowed_quota array. Send a request and verify the client receives a 429.
Test Service Failure: Stop the mock gRPC service entirely or block the port. Send a request and verify the client receives a 503.
Inspect Logs: Run Envoy with debug logging to see the filter's decision process:
envoy --config-path envoy.yaml --log-level debug

Assumptions and Uncertainty
This behavior assumes you are using the default rate_limit filter configuration. There is a possibility of 503s occurring if you have implemented a custom response_code mapping within your filter configuration that explicitly maps denials to 503, though this is not the standard behavior.

One diagnostic detail needed: Are you using any custom failure_mode_deny or response_code overrides in your rate_limit_service configuration?

Does Envoy trigger 503 errors when the rate limit service returns an empty allow list?

1 answer

Answer

Likely Explanation of Behavior

Verification Steps

Assumptions and Uncertainty

Your perspective belongs here.

0 question comments