Nim 2.x concurrent tail-latency attribution limits between ORC and asyncdispatch
0 reputation · 25 Jun 2021, 08:24 UTC
In a Nim 2.x async HTTP service, tail-latency percentiles sometimes degrade only when many requests are in flight. The goal is to attribute those spikes to a specific documented mechanism before tuning the deployment. Two plausible sources are the global ORC cycle-collection pass, which runs under the default memory manager in the 2.x line and has no published pause-time bound, and the single-threaded asyncdispatch event loop, whose documentation warns that blocking calls inside async procedures stall every other request served by that loop.
The uncertainty is compounded by deployment shape: the standard library does not provide one shared multi-core async runtime, so scaling across cores requires --threads:on with one event loop per thread or a third-party framework that may replace asyncdispatch entirely. Because Nim 1.x releases default to refc rather than orc, the active memory manager is also version-sensitive.
Given identical concurrent load, what measurement or instrumentation distinguishes an ORC collection pause from event-loop blocking in a stock Nim 2.x build? Which documented compiler or runtime flags expose the minimum data needed to separate those two causes without relying on third-party profilers? If the service is built with --threads:on, does per-thread loop isolation change the attribution model?