Architecture Note: Using NGINX limit_req for Edge‑Side Rate Limiting

Requirements

The primary goal is to protect upstream services from overload by limiting the number of client requests that reach them per second. The limit must be enforced at the edge before traffic is proxied, allowing upstream applications to assume they receive only throttled traffic. The solution should be operable with a single NGINX instance, require minimal external dependencies, and provide observable metrics for alerting.

Smallest Suitable Design

A single NGINX listener on port 80 (HTTP) or 443 (HTTPS) terminates TLS, defines an upstream block, and applies limit_req in the server or location context before proxy_pass. The design uses the built‑in limit_req_zone directive to allocate a shared memory zone that tracks request timestamps.

# /etc/nginx/nginx.conf (or included file)
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

http {
    upstream app {
        server 10.0.1.10:8080;
    }

    server {
        listen 443 ssl;
        ssl_certificate     /etc/nginx/certs/example.crt;
        ssl_certificate_key /etc/nginx/certs/example.key;

        location /api/ {
            limit_req zone=api_limit burst=20 nodelay;
            proxy_pass http://app;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        # optional status page for monitoring
        location = /nginx_status {
            stub_status on;
            allow 127.0.0.1;
            deny all;
        }
    }
}

Key elements:

• limit_req_zone creates a 10 MB shared zone named api_limit that tracks the binary remote address.
• The zone’s rate is 10r/s (10 requests per second) with a burst of 20, meaning NGINX will allow short spikes up to 30 requests before rejecting excess.
• The limit_req directive is placed before proxy_pass so that rejected requests never reach the upstream.
• NGINX must be reloaded (sudo nginx -s reload) after editing the configuration; this operation requires root or sudo privileges.

Trust and Data Boundaries

NGINX sits at the trust boundary: it is the only component that sees the raw client address and enforces the rate limit. Downstream services trust that any request they receive has already been vetted by NGINX, so they can omit their own per‑IP rate‑limiting logic. No client‑affecting data (e.g., cookies, headers) is altered by the limit_req module; it only decides whether to pass or reject the request.

Operational Checks

To verify that the limit is active and to detect exhaustion:

• Stub status: Access http://localhost/nginx_status (or the configured status endpoint) and watch the 5xx counter rise when requests are rejected.
• Error log: Messages like limiting requests, excess: 20.000 by zone "api_limit" appear in /var/log/nginx/error.log when the burst is exceeded.
• Shared‑zone usage: With NGINX Plus, the API endpoint /api/6/http/limit_req_zones shows current usage. In the open‑source build, the variable $limit_req_status can be logged via a custom log format to observe 0 (accepted) or 1 (rejected).
• Synthetic traffic: Use a tool such as wrk -t2 -c100 -d30s http://your-host/api/ and verify that responses beyond the configured rate return HTTP 503 while earlier requests are proxied with a 2xx status.

All checks should be performed with a non‑privileged user for the traffic generator; only the NGINX reload step needs elevated rights.

Failure Modes

• Zone exhaustion: If the sustained request rate exceeds rate + burst for longer than the zone can absorb, NGINX returns 503 for every excess request. Upstream sees zero traffic during the overload, which is safe but may cause client‑visible errors.
• Reload failure: A syntax error in the new configuration prevents NGINX from loading the updated zone, causing the old configuration to remain active. If the reload is attempted with a broken config, the service may continue with stale limits or, if the old config is missing the zone, allow unlimited traffic.
• Mis‑sized burst: Setting burst too low results in premature 503 responses during legitimate spikes; setting it too high reduces protection, allowing upstream to see bursts larger than intended.
• Memory pressure: An overly large zone=…:size consumes RAM unnecessarily; too small a size can cause the zone to fill quickly, leading to frequent 503s even when the average rate is within limits.

Conditions That Would Change the Design

The single‑node, shared‑memory approach is sufficient when:

• Traffic enters through a single NGINX instance.
• Rate limits are based on a single key (e.g., client IP).
• Operational overhead of reloading a single process is acceptable.

If any of the following arise, the design must be revisited:

1. Multi‑node deployment: Multiple NGINX instances behind a load balancer would each maintain independent zones, causing inconsistent limits. A solution would require NGINX Plus zone synchronization, a central Redis‑backed store, or an external rate‑limiting service (e.g., Envoy, Kong).
2. Per‑API‑key or per‑token limits: The current zone uses $binary_remote_addr. To limit by API key, the zone key would become $http_x_api_key or a variable derived from JWT claims, necessitating a separate zone or a more complex map.
3. Need for sub‑second granularity: The limit_req module works with requests per second; if millisecond‑level bursting is required, alternative mechanisms (e.g., Lua scripting with limit_req‑like logic or an external token bucket) become necessary.
4. Regulatory auditing: If detailed per‑client request logs are required for compliance, the simple 503 response may be insufficient; integrating with a logging pipeline that records both allowed and rejected requests would be needed.

Example Configuration (with placeholders)

Replace the placeholders with values appropriate for your environment:

# Replace these values
limit_req_zone $binary_remote_addr zone=: rate=;

http {
    upstream  {
        server :;
    }

    server {
        listen 443 ssl;
        ssl_certificate     ;
        ssl_certificate_key ;

        location // {
            limit_req zone= burst= nodelay;
            proxy_pass http://;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        location = /nginx_status {
            stub_status on;
            allow 127.0.0.1;
            deny all;
        }
    }
}

After editing, test the configuration with sudo nginx -t and, if successful, reload: sudo nginx -s reload. A failed test will keep the old configuration running, preventing accidental downtime.

Limitations and Practical Verification

The limit_req module only throttles at the HTTP request layer. It does not mitigate:

• Slow‑loris or other connection‑exhaustion attacks (use limit_conn and appropriate client_body_timeout/client_header_timeout).
• Large payloads that could overwhelm upstream buffers (adjust client_body_buffer_size and proxy_buffering).
• Application‑level abuse such as credential stuffing that stays within the request rate.

To confirm the module is active in a staging environment:

1. Enable debug logging temporarily: error_log /var/log/nginx/debug.log debug;.
2. Generate a steady stream of requests at exactly the configured rate (e.g., using hey -z 1m -q 10 -c 1 http://host/api/ for 10 r/s). Expect a mix of 2xx and occasional 503 if burst is exceeded.
3. Check the error log for limiting requests lines and verify that the count matches the expected number of rejections.
4. Disable debug logging after verification to avoid log‑volume overhead.

Remember that any change to the zone size, rate, or key requires a reload; plan reloads during low‑traffic windows or use a load‑balancer to drain connections before reloading.