Rate Limit Service failure mode when rate-limit backend is unreachable

Question

A design is being evaluated that uses Envoy's Rate Limit Service to bound query rates and relies on upstream or filter logic for pagination of large datasets.

RLS is configured via the HTTP connection manager and communicates with an external rate-limit server over gRPC. When a limit is exceeded Envoy can return rate-limit headers such as x-envoy-ratelimited and x-envoy-rate-limit-headers. Envoy does not automatically paginate responses; pagination must be implemented by the upstream service or by a Lua or Wasm filter that interprets paging headers and slices the response.

An unresolved decision concerns the failure mode of RLS when the rate-limit backend is unreachable or slow, and how that interacts with request buffering and retry policies for large bodies.

What failure mode is applied when the rate-limit server cannot be reached? How does the chosen failure mode interact with buffer limits that may affect large request bodies before a rate-limit decision? Is native response pagination supported by Envoy, or must pagination always be delegated to upstream or a filter?

Tasadduq Burney · Answer

Failure Mode Behavior
When the Rate Limit Service (RLS) gRPC backend is unreachable or times out, Envoy's behavior is governed by the failure_mode_allow configuration setting:

false (Default): Envoy denies the request, returning a 429 Too Many Requests response with the x-envoy-ratelimited header.
true: Envoy allows the request to proceed to the upstream service despite the backend failure.

Interaction with Request Buffering
The rate-limit decision occurs before Envoy buffers the request body. This creates two distinct paths for large request bodies:

In Deny Mode (failure_mode_allow: false): The request is rejected immediately. Because the body is not yet read, large payloads will not trigger the max_request_body_kb limit; the client receives a 429.
In Allow Mode (failure_mode_allow: true): The request proceeds to the buffering stage. If the body exceeds the max_request_body_kb limit defined in the HTTP connection manager, Envoy returns a 413 Payload Too Large before the request reaches the upstream.

Response Pagination
Envoy does not provide native response pagination. To handle large datasets, pagination must be implemented in one of the following ways:

Upstream Service: The backend service manages offsets/cursors and returns paginated chunks.
Custom Filters: A Lua or Wasm filter is used to interpret paging headers and slice the response body.

Verification and Diagnostics

Config Audit: Verify the failure_mode_allow value and the gRPC timeout in the rate_limit_service block.
Outage Simulation: Block the gRPC port to the RLS backend. A 429 confirms deny mode; a 2xx (or upstream error) confirms allow mode.
Buffer Test: With failure_mode_allow: true, send a payload exceeding max_request_body_kb to verify the 413 response.
Log Analysis: Use --log-level debug to identify "rate limit service unavailable" events.

To refine this recommendation: is the current failure_mode_allow set to true or false? This determines whether your primary risk is unintended traffic blockage (429) or buffer-related rejections (413).

Rate Limit Service failure mode when rate-limit backend is unreachable

1 answer

Failure Mode Behavior

Interaction with Request Buffering

Response Pagination

Verification and Diagnostics

Your perspective belongs here.

0 question comments