NGINX Reverse Proxy Architecture: Minimal Upstream Design with Keepalive and Least‑Conn Load Balancing

Requirements

Place a single NGINX instance in front of two or more application servers. The proxy must terminate TLS, forward the real client IP, reuse connections to back‑ends, and distribute load based on current connection count rather than a static rotation.

Smallest Suitable Design

An upstream block that lists the backend servers together with a proxy_pass directive in a location is the minimal load‑balancing configuration. The default algorithm is weighted round‑robin; switching to least_conn makes the proxy send each request to the server with the fewest active connections, which is useful when request processing time varies.

upstream app_backend {
    least_conn;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    keepalive 32;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate     /etc/nginx/certs/api.example.com.crt;
    ssl_certificate_key /etc/nginx/certs/api.example.com.key;

    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_next_upstream error timeout http_500 http_502 http_503 http_504;
        proxy_next_upstream_tries 2;
    }
}

Why each directive matters

• least_conn – chooses the backend with the fewest active connections.
• keepalive 32 – keeps up to 32 idle HTTP/1.1 connections per worker process to each upstream server, avoiding repeated TCP/TLS handshakes.
• proxy_http_version 1.1 and proxy_set_header Connection "" – required for keepalive to work; they tell NGINX to send HTTP/1.1 without a "close" header.
• X-Forwarded-For / X-Real-IP – convey the original client address to the application.
• proxy_next_upstream – defines which error conditions trigger a retry on another backend. The example retries only on idempotent‑safe failures and limits retries to two attempts.

Trust and Data Boundaries

NGINX is the trust boundary. TLS terminates here, so the internal network never sees encrypted traffic. Backend services must only trust X-Forwarded-For and X-Real-IP when the request arrives from the proxy’s IP address. A common hardening step is to configure the application firewall (or the app itself) to reject direct connections that lack these headers or that come from any IP other than the proxy.

Operational Checks

1. Configuration test – run nginx -t as root (or with sudo) on the proxy host. Expected output: syntax is ok and test is successful.
2. Reload – systemctl reload nginx (or service nginx reload) applies changes without dropping existing connections.
3. Verify distribution – send concurrent requests (e.g., for i in {1..100}; do curl -s -o /dev/null -w "%{http_code}\n" https://api.example.com/health; done | sort | uniq -c) and inspect each backend’s access log. You should see roughly equal counts when load is uniform, or a skew toward the less‑busy server when request latency differs.
4. Confirm keepalive reuse – on a backend, run ss -tan state established '( dport = :8080 )' | wc -l before and after a load burst. The connection count should stay near the keepalive limit rather than climbing with each request.
5. Health‑check simulation – stop one backend (systemctl stop myapp) and watch the proxy’s error log. After max_fails (default 1) failures within fail_timeout (default 10s) the server is marked down and traffic shifts to the remaining host. Restart the backend and observe automatic recovery.

Failure Modes and Design Adjustments

Failure mode	Symptom	Mitigation / design change
All upstreams marked unavailable	Clients receive 502 Bad Gateway	Add a third backend or a hot‑standby; configure a custom error_page 502 /maintenance.html for graceful degradation.
Slow backend exhausts worker connections	NGINX logs worker_connections are not enough or latency spikes	Increase worker_connections or set proxy_read_timeout lower; consider proxy_buffering off for streaming responses.
Retrying non‑idempotent requests (POST, PATCH)	Duplicate side‑effects (e.g., double charge)	Restrict proxy_next_upstream to idempotent methods only, or add proxy_next_upstream_tries 1 for unsafe methods.
Backend IP spoofing	Application trusts X-Forwarded-For from untrusted source	Enforce that only the proxy’s IP may set these headers (e.g., set_real_ip_from 10.0.0.5/32; real_ip_header X-Forwarded-For; on the backend).

Conditions That Would Change the Design

• Need for active health checks – the open‑source NGINX only supports passive checks (max_fails/fail_timeout). If you require HTTP‑level probes, adopt NGINX Plus or an external checker (e.g., Consul, HAProxy sidecar).
• Highly heterogeneous request costs – if some endpoints are dramatically heavier, consider adding weight parameters to the upstream servers or moving to a separate upstream pool per service.
• TLS offload to a dedicated terminator – when NGINX runs on a constrained host, you might terminate TLS on a hardware load balancer and have NGINX operate purely as a Layer‑7 router.
• Version‑specific keepalive quirks – directives such as proxy_http_version and header handling changed between 1.14 and 1.24. Always validate the exact syntax against the deployed version (nginx -V).

Limitations

The configuration shown works on NGINX 1.18+ (the keepalive directive inside upstream was stabilized in 1.15). Active health checks, dynamic reconfiguration via API, and advanced metrics require the commercial NGINX Plus subscription. The proxy_next_upstream retry logic cannot distinguish between idempotent and non‑idempotent methods without explicit method checks; you must duplicate the location block or use if (discouraged) to separate them.

Practical Verification Checklist

• Run nginx -t and reload.
• Generate load and confirm backend logs show balanced distribution.
• Inspect backend connection counts to verify keepalive reuse.
• Simulate a backend failure and watch automatic failover.
• Confirm backends reject direct traffic lacking proxy‑set headers.

Following this minimal, well‑established pattern gives you a production‑grade reverse proxy with connection reuse, sensible load distribution, and clear failure semantics—without pulling in unnecessary complexity.