Question
Increased tail latency under concurrent load with tRPC httpBatchLink
Ira Maple
0 reputation · 29 Apr 2026, 07:12 UTC
68.7K views1
Loading…
0 reputation · 29 Apr 2026, 07:12 UTC
When using tRPC's httpBatchLink with the default 1 ms batchWindow, request latency remains low for isolated calls but shows a noticeable increase in the 95th‑plus percentile as concurrent request volume rises.
The batchWindow is a fixed timer that does not adapt to request arrival patterns, and any synchronous work in middleware or context creation can delay the event‑loop tick that triggers the batch flush, making it hard to isolate whether the observed tail latency stems from the batching window itself, event‑loop saturation, or a combination of both. Without built‑in metrics for batch fill rate or per‑request queue time, diagnosing the root cause relies on external instrumentation.
What batchWindow value reduces the observed tail latency for a given concurrency level? How does synchronous middleware amplify batch flush jitter under load? Is there a lightweight way to expose per‑request enqueue time without adding custom middleware?
To reduce observed tail latency (P95+) caused by batching, set the batchWindow to 0. This effectively disables the batching delay, ensuring requests are dispatched immediately rather than waiting for the timer to expire.
The httpBatchLink uses a timer-based mechanism to group requests. While a 1ms window seems negligible, it introduces a mandatory minimum delay for the first request in any batch. Under concurrent load, this behavior is amplified by the Node.js event loop:
trpcbatchWindow plus the duration of any blocking synchronous tasks, leading to the spikes seen in high-percentile latency.Depending on your performance goals, choose one of the following configurations:
Use this if your server can handle a higher volume of HTTP requests and your primary goal is minimizing tail latency.
const link = httpBatchLink({
url: 'http://localhost:3000/trpc',
batchWindow: 0, // Disables the waiting window
});If reducing the number of requests is more critical than P99 latency, keep a small window but move all synchronous logic (e.g., heavy validation or synchronous crypto) into asynchronous wrappers or separate worker threads to keep the event loop responsive.
batchWindow: 0. If the tail latency disappears, the issue was the batching timer.perf_hooks in your Node.js environment to track event loop lag. Correlate lag spikes with the timing of your tRPC batch flushes.tRPC does not provide a native, lightweight property for per-request enqueue time. To track this without custom middleware, you must wrap the httpBatchLink in a custom Link that captures a timestamp immediately before calling the next link in the chain and compares it to the timestamp when the response is received.
Diagnostic Detail Needed: Are you observing these latency spikes on the client-side (time to dispatch) or the server-side (time to process)? If the lag is server-side, adjusting the client batchWindow will not resolve the root cause.
Use comments to ask for clarification. Post a solution as an answer.
6,510 reputation · 29 Apr 2026, 13:25 UTC
The 1 ms batchWindow relies on setTimeout, which in Node.js has a minimum granularity of ~1 ms but can drift to 2–4 ms under event-loop pressure. When synchronous middleware runs during that window, it doesn't just add its own duration—it pushes the timer callback to a later macrotask tick, so every request in the batch absorbs the full delay. This effect compounds at higher concurrency because more middleware executions contend for the same tick.
A practical way to quantify the queue-time component without custom middleware is to wrap the link's batch function and record process.hrtime.bigint() at enqueue and at flush; the difference isolates wait time from network and handler latency. Compare distributions across batchWindow values (0.5, 1, 2, 5 ms) at your target concurrency to find the elbow where tail latency stops improving.