Scalingo router concurrent request queuing under container saturation

Answer

Scalingo's HTTP router does not expose documented per-container concurrent request limits or explicit queuing behavior. When web containers saturate, the platform router distributes traffic via load balancing, but any queuing that occurs is not publicly specified. Latency under concurrent load is more likely to originate inside your application container than at the router level.

Confirmed Platform Behavior

• The router load-balances requests across multiple web containers using round-robin or similar algorithms.
• No public documentation specifies a hard concurrent request limit per container.
• Queuing behavior, if present, is implementation detail not exposed to users.
• Platform timeouts and backpressure mechanisms are not documented in public resources.

Where Latency Actually Comes From

In practice, latency under concurrent load typically stems from application-level bottlenecks:

1. Database connection pools reaching configured limits
2. Application code with synchronous operations or memory pressure
3. External API rate limits or downstream service saturation
4. Container resource limits (CPU/memory) causing throttling

Diagnostic Steps to Isolate the Source

To determine whether latency originates at the router or container level:

1. Enable Scalingo metrics in the dashboard and monitor container CPU/memory during load tests
2. Check application logs for connection pool exhaustion or request timeouts
3. Run controlled load tests with gradually increasing concurrency while measuring response times
4. Correlate platform metrics (container utilization) with application-level metrics (database pool usage)

Recommended Actions

• Scale horizontally if container CPU consistently maxes out during load
• Increase database connection pool size if pool exhaustion appears in logs
• Add application-level request queuing with timeouts if external dependencies are the bottleneck
• Use health checks and readiness probes to prevent router-level queuing of unhealthy instances

Key Missing Detail

What is your current plan type and region? Scalingo's behavior may vary between starter, business, and enterprise plans, and some regions may have different routing configurations.