Choosing the Right Load‑Balancing Algorithm in NGINX Open Source

The Decision Problem

When you expose a pool of backend servers behind a single NGINX front‑end, you must decide how that traffic is distributed. The wrong choice can leave some nodes under‑utilised while others become bottlenecks, especially when request sizes vary widely.

The core decision is: Which traffic‑distribution algorithm best matches the workload and infrastructure?

Algorithm Options in NGINX Open Source

Algorithm	How It Works	Ideal Use‑Case	Primary Risk
Round Robin (default)	Sequentially sends each new request to the next server in the list.	Uniform, short‑lived requests where all servers are equally capable.	Can overload a node if some requests are longer than others.
Least Connections (least_conn)	Routes to the server with the fewest active connections at the moment.	Workloads with variable processing times or long‑lived connections.	“Herd behaviour” when a new server starts with zero connections.
IP Hash (ip_hash)	Deterministically maps a client’s IP address to a specific server.	Session persistence when the application stores session data locally.	Uneven load if many clients share the same source IP (e.g., corporate proxy).

Trade‑Offs and Constraints

Round Robin + Weight

NGINX allows a weight attribute on each server directive. A server with weight 3 will receive roughly three times as many requests as a server with weight 1. This is useful when the pool contains heterogeneous hardware.

Least Connections + Weight

The least_conn directive can also be combined with weight, letting a stronger node absorb a larger share of the load while still considering current connections.

IP Hash Limitations

Because IP hash is purely deterministic, it cannot be combined with least_conn or weight. If your clients frequently change IPs (mobile, VPN, NAT), session persistence will break.

Herd Behaviour Mitigation

When a new server is added, it starts with zero connections and may receive a burst of traffic. A common mitigation is to start the new service slowly or use the max_fails and fail_timeout directives to temporarily mark it as unhealthy.

Concrete Implementation

The following example shows an upstream pool that mixes a powerful node with two standard nodes, uses least_conn to balance load, and applies weight to the powerful node.

# /etc/nginx/conf.d/backend.conf
# Requires root or sudo to edit and reload NGINX

http {
    upstream backend_pool {
        # Choose the server with the fewest active connections
        least_conn;

        # Powerful node – weight 3
        server 10.0.0.10:8080 weight=3;

        # Standard nodes – weight 1 each
        server 10.0.0.11:8080 weight=1;
        server 10.0.0.12:8080 weight=1;
    }

    server {
        listen 80;
        server_name example.com;

        location /api/ {
            proxy_pass http://backend_pool;
            # Add a header that shows which backend handled the request
            add_header X-Upstream-Server $upstream_addr always;
        }
    }
}

Explanation of key directives:

• least_conn – dynamic load balancing based on active connections.
• weight=3 – this server will receive roughly three times as many requests as a weight‑1 server.
• add_header ... always – ensures the header appears on all responses, even non‑2xx, which is necessary for verification.

Validation Strategy

1. Test the configuration syntax:

   nginx -t

2. Reload NGINX:

   systemctl reload nginx

3. Simulate traffic – from a client machine or a load‑generator:

   # Simple loop that prints the X-Upstream-Server header
   for i in {1..20}; do
     curl -s -o /dev/null -w "%{http_code} %{upstream_addr}\n" http://your-nginx-host/api/
   done
   

4. Check backend connection counts – on each server, run:

   ss -tanp | grep LISTEN | grep -c "LISTEN"
   # or use netstat -anp | grep LISTEN
   

   This gives a rough count of active connections; you should see the weighted server handling more connections.
5. Observe for herd behaviour – if you add a new server with zero weight, you may see a spike. In that case, pause the new server for a minute before fully exposing it.

Rollback Procedure

If the new configuration causes 502 Bad Gateway errors or other issues, revert to the default Round Robin algorithm:

1. Edit the upstream block to remove least_conn (and any weight values if no weighting is needed).
2. Run nginx -t to verify syntax.
3. Reload NGINX with systemctl reload nginx.
4. Confirm traffic is balanced evenly by repeating the validation steps.

Because the change only affects routing logic, the rollback does not alter any persistent data or configuration outside the upstream block.

When to Use Which Algorithm

• Round Robin – all requests are short and similar; no session persistence required.
• Least Connections – request durations vary, long‑lived connections (WebSockets, streaming), or when you want to avoid overloading a busy node.
• IP Hash – you need sticky sessions and do not have a shared session store; be aware of NAT or corporate proxies.

Always test the chosen algorithm in a staging environment that mirrors production traffic patterns before promoting to production.