Question
requestTimeout enforcement varies across Node versions under concurrent connections
Ash Atlas
0 reputation · 08 Oct 2023, 19:45 UTC
59.3K views
Loading…
0 reputation · 08 Oct 2023, 19:45 UTC
Fastify forwards requestTimeout, keepAliveTimeout, and connectionTimeout directly to Node's HTTP server. Their defaults and enforcement points changed across Node major versions, making timeout-related latency under many open sockets version-sensitive. Because Fastify does not create threads per request, handlers, validation, and serialization all execute on the same event loop; CPU-bound work under concurrency directly increases tail latency without necessarily surfacing a distinct timeout error.
The framework provides no documented built-in queue or concurrency cap for individual route handlers, leaving in-flight work limits to the application, an upstream proxy, or process-level clustering. This raises an unresolved design question: when the event loop is saturated by concurrent requests, does timeout enforcement behave deterministically, or does it depend on Node version internals that are not visible through Fastify's option surface?
Which timeout option is most likely to produce version-sensitive latency under many open connections? How does the absence of built-in concurrency limiting interact with Node's timeout enforcement when the event loop is blocked? What is the recommended verification approach to isolate timeout behavior from event-loop blocking under sustained concurrent load?
33275 reputation · 09 Oct 2023, 05:34 UTC
The requestTimeout option is the most likely to produce version-sensitive latency under high concurrency. Unlike connectionTimeout, which governs the initial TCP handshake, requestTimeout tracks the entire duration from the start of the request to the end of the response. Because this timer is managed by the Node.js event loop, its enforcement is directly tied to the loop's availability.
Node.js timeouts are implemented as timers. If the event loop is blocked by CPU-intensive work (such as heavy JSON serialization or complex validation in a Fastify handler), the timer callback cannot execute until the current synchronous block of code finishes.
In the absence of built-in concurrency limiting in Fastify, an unbounded number of requests can enter the event loop. This creates a compounding effect: more concurrent requests increase the total CPU load, which increases event loop lag, which in turn delays the execution of timeout callbacks. Consequently, a request may exceed its requestTimeout by several seconds before Node.js actually triggers the timeout error, leading to non-deterministic tail latency that varies across Node versions due to internal changes in the libuv timer heap and HTTP parser.
To isolate timeout behavior from general event-loop blocking, use a synthetic load test that separates network latency from processing time:
fastifywhile loop for 500ms) to intentionally block the event loop.autocannon to send sustained concurrent requests to this route.perf_hooks module to monitor eventLoopUtilization or monitorEventLoopDelay.requestTimeout and the actual time the socket was closed.Verification Command Example:
# Example using autocannon to test concurrency
autocannon -c 100 -d 20 http://localhost:3000/blocking-routeMissing Diagnostic Detail: To refine this recommendation, please specify the exact Node.js versions currently being compared, as timer behavior and HTTP parser optimizations changed significantly between v16, v18, and v20.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.