Load Balancing Across Backends with NGINX upstream

Useful answer

If one application server handles every request, traffic spikes and server crashes become site outages. The practical fix is to run NGINX as a reverse proxy in front of a small backend pool and use the upstream module to distribute requests. NGINX accepts the client connection, selects a backend from the pool, forwards the request, and temporarily stops sending traffic to a backend that stops responding. The default method is round robin; least_conn sends new requests to the server with the fewest active connections.

How the upstream module works

An upstream block names a group of backend servers. A proxy_pass directive in a server or location block references that name instead of a single address. NGINX tracks connections and failures per backend.

Key terms:

• Round robin: the default algorithm. NGINX hands requests to servers in order, one at a time.
• least_conn: a directive inside upstream that chooses the backend with the fewest active connections.
• weight: a parameter on a server line that sends proportionally more traffic to higher-capacity hosts.
• max_fails and fail_timeout: passive health-check parameters. If a backend fails max_fails times within fail_timeout, NGINX marks it unavailable for the rest of that window.
• proxy_set_header: forwards client details such as the original IP and Host header, because backends otherwise see NGINX as the client.

Worked configuration

This example runs on the NGINX host and balances between two local backends on ports 8081 and 8082. Place it in /etc/nginx/nginx.conf or a site file under /etc/nginx/sites-available/, then enable it with a symlink in sites-enabled/ if your distribution uses that layout.

upstream app_pool {
    least_conn;
    server 127.0.0.1:8081 weight=3 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8082 max_fails=3 fail_timeout=30s;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://app_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 60s;
        proxy_next_upstream error timeout http_502 http_503 http_504;
    }
}

Run the following on the NGINX host with root or sudo privileges:

1. Back up the current configuration: sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
2. Edit the file and insert the upstream and server blocks above, replacing 127.0.0.1:8081 and 127.0.0.1:8082 with your real backend addresses.
3. Validate syntax without applying changes: sudo nginx -t. Expected result is a success message; any syntax error must be fixed before reloading.
4. Apply the configuration: sudo systemctl reload nginx. This changes running state. If reload fails or behavior is wrong, restore the backup with sudo cp /etc/nginx/nginx.conf.bak /etc/nginx/nginx.conf and run sudo systemctl reload nginx again.

Risk: a syntax error prevents the reload and leaves the old configuration running. A proxy_read_timeout that is too short makes NGINX return 504 while a slow backend is still working.

Verifying traffic distribution

Run two test backends on ports 8081 and 8082, then send repeated requests through NGINX:

for i in $(seq 1 10); do
  curl -s -o /dev/null -w "request $i: %{http_code}\n" http://127.0.0.1/
done

Check the backend access logs to confirm requests landed on both servers. To see which upstream handled each request in NGINX’s own log, add $upstream_addr to your log_format and watch /var/log/nginx/access.log while curling.

To verify passive failure handling, stop one backend process. After up to max_fails failures within fail_timeout (30 seconds in the example), NGINX marks that backend unavailable and sends new requests to the remaining server. When the backend returns, NGINX gradually returns it to the pool after the timeout window passes.

Limits and common mistakes

Directive	Behavior	Best for
Round robin (default)	Distributes requests sequentially	Backends with similar capacity
least_conn	Sends traffic to the backend with fewest active connections	Backends with uneven response times
weight	Sends a proportional share of requests	Uneven hardware capacity

• Open-source NGINX performs passive health checks only. It detects failures by observing failed requests; it does not actively probe backends. Active health checks require NGINX Plus.
• ip_hash sticks a client to one backend, but a few busy clients can overload that server while others sit idle.
• proxy_next_upstream defaults to retrying only idempotent methods such as GET and HEAD. A POST that fails on one backend may return 502 to the client even if another backend is healthy. Explicitly list retry conditions as shown above.
• weight changes proportions, not guarantees. Do not expect an exact 3:1 split on a small number of requests.
• max_fails and fail_timeout are passive. A backend must fail the configured number of times within the window before NGINX stops using it.

Practical check

Run sudo nginx -T to print the effective configuration and confirm the upstream block and directives are active. Then generate traffic and inspect NGINX access logs containing $upstream_addr. If every line shows the same backend, check whether ip_hash is enabled or whether one backend is failing and being skipped.

This behavior applies to NGINX Open Source 1.x. Module availability depends on how your NGINX binary was compiled; nginx -V lists build options.