DigitalOcean Load Balancer latency during high concurrent TCP connections

DigitalOcean does not publicly disclose a specific, static numerical value for the internal connection queue depth limit per backend for their managed Load Balancers. Because this is a managed service, the proxy layer's internal queuing mechanisms are abstracted from the user and cannot be adjusted via the Control Panel or API.

Likely Cause of Latency

While the Load Balancer manages the initial entry, latency during high TCP concurrency is frequently caused by a mismatch between the Load Balancer's delivery rate and the backend Droplet's ability to accept new connections. If the backend's TCP listen queue is full, the Load Balancer may hold the connection in its own internal buffer or retry the connection, which manifests as increased latency to the end user.

Confirmed Backend Bottlenecks

• Kernel Listen Queue: The net.core.somaxconn setting on the backend Droplet limits the number of queued connections that the kernel will hold before the application accepts them.
• Application Backlog: The listen() system call in your application code defines its own backlog limit; if this is lower than the kernel limit, it becomes the primary bottleneck.
• Connection Churn: Frequent TCP handshakes (due to disabled or short keep-alive timeouts) increase overhead and can saturate the proxy-to-backend path faster than persistent connections.

Recommended Mitigation Steps

Since you cannot modify the Load Balancer's internal queue, you must optimize the backend to ensure it clears the Load Balancer's queue as quickly as possible.

1. Increase Kernel Queue Depth: Raise the maximum socket listen queue on your backend Droplets to prevent the OS from dropping or delaying SYN packets.

   sudo sysctl -w net.core.somaxconn=1024

   Note: Persist this by adding net.core.somaxconn=1024 to /etc/sysctl.conf.
2. Align Keep-Alive Settings: Ensure your application's keep-alive timeout is slightly longer than the Load Balancer's timeout to prevent the LB from attempting to use a connection the backend has already closed.
3. Scale Horizontally: Because the queue is per-backend, adding more Droplets to the Load Balancer pool distributes the concurrent connection load, effectively increasing the total aggregate queue capacity.

Verification and Monitoring

To determine if the latency is occurring at the Load Balancer or the Droplet, run the following command on the backend during a load spike:

ss -nlt

Check the Send-Q column for your application port. If Send-Q is equal to the Recv-Q (the limit), your backend is saturated, and the Load Balancer is forced to queue requests.

Missing Diagnostic: Are you using Layer 4 (TCP) or Layer 7 (HTTP) load balancing? Layer 7 proxying introduces different buffering behaviors and header processing overhead that may shift the bottleneck.