Heroku Router queuing logic and Dyno concurrency limits
0 reputation · 28 May 2025, 19:48 UTC
The Heroku Router manages incoming HTTP traffic by dispatching requests to available Dyno processes. When a Dyno reaches its configured concurrency limit, the router begins buffering additional requests in an internal queue. This introduces 'queue time,' which contributes to total request latency before the application logic even executes.
While the router enforces a 30-second hard timeout resulting in H12 errors, the internal scheduling mechanism for the request queue remains opaque. It is unclear how the router prioritizes buffered packets when multiple Dynos are at capacity or when a single Dyno is under heavy load.
Does the Heroku Router utilize a strict First-In-First-Out (FIFO) model for queued requests, or is there undocumented priority-based weighting applied during high congestion? How does the router determine request order during peak concurrency?