Using NGINX limit_req to Shield APIs from Traffic Bursts

Problem: Sudden request spikes overwhelm backend services

When a client or a misbehaving script sends a burst of requests faster than your application can handle, you see increased latency, failed responses, and a degraded experience for everyone. Relying solely on application‑level throttling often means the damage is already done by the time the code runs.

Thesis: NGINX’s limit_req module provides a lightweight, configurable leaky‑bucket filter that can reject or delay excess traffic before it reaches your upstream.

How limit_req works

The module creates a shared memory zone that tracks a counter per key (e.g., IP address). It allows a steady rate of requests per second and lets you accumulate a burst of extra tokens. When the token bucket is empty, NGINX either returns a status code (default 503) or, if delay is used, holds the request until a token becomes available.

Worked example: protecting an API endpoint

Assume you have an API under /api/ proxied to a backend upstream. You want to allow each client IP to sustain 10 requests per second, with the ability to handle short bursts of up to 20 additional requests without queuing.

# http block – define the shared zone
limit_req_zone $binary_remote_addr zone=api_limits:10m rate=10r/s;

# server block – apply the zone to the location
server {
    listen 80;
    server_name example.com;

    location /api/ {
        limit_req zone=api_limits burst=20 nodelay;
        proxy_pass http://backend_upstream;
    }
}

limit_req_zone creates a 10 MB zone named api_limits that can store state for roughly 800 k distinct IP keys (each entry ~64 bytes). The rate of 10r/s sets the refill speed of the token bucket. The burst=20 lets the bucket hold up to 20 extra tokens; nodelay tells NGINX to reject immediately when the bucket is empty rather than waiting.

Trade‑offs and limitations

• Immediate rejection vs. smoothing: Using nodelay protects the backend from sudden overload but can return 503 errors to legitimate clients during a genuine spike. If you prefer a smoother experience, replace nodelay with delay (or omit it) so excess requests are queued until tokens become available.
• Zone sizing: An undersized zone may cause NGINX to drop new keys when memory is exhausted, leading to unexpected limit bypass. Estimate the number of unique clients you expect (e.g., 50 k) and multiply by ~64 bytes per entry, adding a safety margin.
• Granularity: The example limits by IP address. If many clients share a NAT or you need per‑API‑key limits, change the key (e.g., $api_key from a header) and adjust the zone size accordingly.

Verification steps

1. Reload configuration: nginx -s reload (requires write access to the NGINX config and permission to signal the master process).
2. Confirm the zone: Run nginx -T and look for the line limit_req_zone $binary_remote_addr zone=api_limits:10m rate=10r/s; in the output.
3. Generate test traffic: From a client machine, use a tool like wrk:

   wrk -t12 -c400 -d30s http://example.com/api/

   Observe the responses; requests beyond the allowed rate should return HTTP 503.
4. Check logs: Enable the limit_req log format (default includes the status code). Look for entries like:

   192.0.2.45 - - [15/Oct/2026:12:34:56 +0000] \"GET /api/resource HTTP/1.1\" 503 0 \"-\" \"wrk\" \"-\"

   The status 503 indicates the request was rejected by limit_req.

Actionable closing

Start with a conservative rate (e.g., 5 r/s) and a modest burst, then monitor your access logs and backend metrics. Adjust the rate and burst values until legitimate traffic flows smoothly while obvious abuse is curtailed. Remember to revisit the zone size whenever you expect a significant change in the number of distinct clients.