Diagnosing and Fixing Hanging Requests in Fastify Applications

Recognizable Condition: Requests That Never Complete

Clients report requests that hang until a timeout fires—either the client's configured timeout or the server's requestTimeout (default 0, meaning no server-side timeout). In logs you see the incoming request line but no corresponding response line. CPU usage may be low or spiking, and memory may grow steadily. The issue affects specific routes or appears globally under load.

Quick Cause–Diagnostic Mapping

Observed Symptom	Likely Root Cause	Primary Diagnostic
Specific route hangs; others work	Async handler missing reply.send() or Promise return	Inspect handler signature; enable debug logging
All routes slow/hang under moderate load	Event loop blocked by synchronous CPU work	Measure event loop delay with clinic.js
Intermittent hangs after adding middleware	Third‑party middleware not awaiting next() or calling reply.send()	Review middleware order and async compliance
Hangs correlate with unhandled promise rejections	Uncaught async exceptions leaving request in limbo	Check process warnings; add global error handler
Latency spikes + rising memory	Plugin memory leak causing GC pressure	Heap snapshots; --trace-gc flag

Ordered Diagnostic Checks

1. Enable request lifecycle logging. Start the server with LOG_LEVEL=debug or set logger: { level: 'debug' } in the Fastify options. Look for the sequence onRequest → preHandler → handler → onSend → onResponse. A missing onResponse entry pinpoints the stalled phase.
2. Verify handler contracts. Every route handler must either return a Promise or call reply.send() exactly once. Example of a broken handler:

   // BAD: neither returns a promise nor calls reply.send()
   fastify.get('/bad', async (request, reply) => {
     const data = await fetchData();
     // forgot reply.send(data)
   });

   Fixed version:

   // GOOD: returns the promise
   fastify.get('/good', async (request, reply) => {
     const data = await fetchData();
     return data; // Fastify sends automatically
   });

   Or explicit reply:

   fastify.get('/also-good', async (request, reply) => {
     const data = await fetchData();
     reply.send(data);
   });

3. Measure event loop delay under load. Run npx clinic doctor -- node server.js (requires @clinic/doctor) while generating traffic with npx autocannon -c 100 -d 30 http://localhost:3000/. An event loop delay consistently above 10 ms indicates synchronous blocking. Look for fs.readFileSync, heavy JSON parsing, or crypto operations in hot paths.
4. Audit third‑party middleware. Middleware wrapped with fastify-plugin must follow the async pattern. A common mistake:

   // BAD: synchronous middleware that forgets next()
   fastify.addHook('onRequest', (request, reply, next) => {
     doHeavySyncCheck(); // blocks loop
     // next() never called
   });

   Correct async middleware:

   fastify.addHook('onRequest', async (request, reply) => {
     await doAsyncCheck(); // or return promise
   });

5. Catch unhandled promise rejections. Add a global handler early in bootstrap:

   process.on('unhandledRejection', (reason, promise) => {
     fastify.log.error({ err: reason }, 'Unhandled rejection');
     // Optionally close server gracefully
   });

   Also ensure fastify.setErrorHandler() is defined so route errors always produce a response.
6. Profile memory if leaks suspected. Run with node --trace-gc --inspect server.js and take heap snapshots in Chrome DevTools after sustained load. Look for retained objects in custom plugins (e.g., caches without eviction, unclosed database connections).

Fixes Tied to Findings

Finding	Fix	Verification
Missing reply.send() / no Promise return	Refactor handler to return value or call reply.send() once	Debug log shows onResponse; autocannon reports 0 timeouts
Sync CPU work in handler	Offload to worker thread (worker_threads) or rewrite as async; use piscina for pool	Clinic doctor shows event loop delay < 5 ms under same load
Non‑compliant middleware	Replace with async version or wrap callback‑based libs via util.promisify	Request lifecycle logs complete for all hooks
Uncaught async errors	Add global setErrorHandler and unhandledRejection listener	No process warnings; every error yields HTTP 5xx with body
Plugin memory leak	Implement cache TTL, close connections in onClose hook, avoid global mutable state	Heap size stabilizes over 1 h load test; GC pauses shrink

Concrete Example: Diagnosing a Stalled /report Endpoint

Assume a route that generates a PDF report. Users report 30 s hangs. Steps:

1. Enable debug logging: LOG_LEVEL=debug npm start.
2. Trigger request: curl -v http://localhost:3000/report.
3. Logs stop at handler—no onSend or onResponse.
4. Inspect handler code:

   fastify.post('/report', async (request, reply) => {
     const pdfBuffer = generatePdfSync(request.body); // CPU‑heavy sync
     reply.header('Content-Type', 'application/pdf');
     // missing reply.send(pdfBuffer)
   });

5. Two problems: synchronous PDF generation blocks the loop, and reply.send() is absent.
6. Fix: move generation to a worker thread and return the buffer.

   const { Worker } = require('worker_threads');
   function generatePdfAsync(data) {
     return new Promise((resolve, reject) => {
       const worker = new Worker('./pdf-worker.js', { workerData: data });
       worker.on('message', resolve);
       worker.on('error', reject);
       worker.on('exit', code => code !== 0 && reject(new Error(`exit ${code}`)));
     });
   }
   
   fastify.post('/report', async (request, reply) => {
     const pdfBuffer = await generatePdfAsync(request.body);
     reply.header('Content-Type', 'application/pdf');
     return pdfBuffer; // Fastify sends buffer automatically
   });

7. Re‑run clinic doctor + autocannon. Event loop delay drops from 200 ms to 3 ms; zero timeouts in 30 s test.

Escalation Criteria

• After applying the above fixes, requests still hang—suspect OS‑level limits (file descriptors, ulimit -n) or reverse proxy timeouts (nginx proxy_read_timeout).
• Memory grows unbounded despite plugin fixes—engage a Node.js core dump analysis (lldb or gcore) to identify native addon leaks.
• Event loop delay remains high with no synchronous code visible—check for excessive microtasks (e.g., thousands of Promise.resolve().then() in a loop) or libuv thread pool exhaustion (UV_THREADPOOL_SIZE).

Limitations & Practical Verification

This guide covers the most common Fastify‑specific patterns. It does not replace full observability (APM, distributed tracing). Always verify fixes with a load test that mirrors production traffic shape (concurrency, payload size, header variety). Use autocannon -c 200 -d 60 -p POST -b '{"data":"..."}' http://localhost:3000/report and confirm 99th‑percentile latency stays below your SLA. If the issue only appears under specific network conditions (e.g., slow clients), enable requestTimeout in Fastify options (e.g., requestTimeout: 30000) to prevent indefinite socket occupation.