Nim 2.x concurrent tail-latency attribution limits between ORC and asyncdispatch

Answer first

In a stock Nim 2.x build you can separate an ORC collection pause from asyncdispatch event-loop blocking with the built-in trace and GC stats, no third-party profiler required. Compile with the default ORC GC and profiling enabled, then compare request latency spikes to the trace intervals for orc_collect/orc_collect_all versus a blocking call on the asyncdispatch stack.

With --threads:on the attribution model changes: each OS thread runs its own asyncdispatch loop and its own ORC state. A pause or block is visible only on the thread that incurred it; other threads continue to serve requests. Without threads, one loop and one GC dominate the process.

Confirmed facts

• Nim 2.x defaults to --gc:orc. Nim 1.x defaults to refc. The active manager is version-sensitive.
• asyncdispatch is single-threaded per thread. The docs warn that a blocking call inside an async proc stalls every other request served by that loop.
• --profile generates a nim.prof trace with entry/exit times for routines, including orc_collect and the asyncdispatch loop.
• -d:gcstats prints a summary of collection counts and total pause time at program exit. It confirms GC is active and gives coarse pause metrics, not per-pause timestamps.
• The stdlib does not provide a shared multi-core async runtime. Scaling across cores requires --threads:on with one event loop per thread.

Likely explanation vs measurement

ORC has no published pause-time bound. Under high concurrency the cycle collector can run for a non-trivial interval, which looks like a tail-latency spike. Event-loop blocking also produces tail spikes, but the trace shape differs: a blocking call appears as the asyncdispatch routine waiting on a user or system call, not as orc_collect on the stack.

Minimum instrumentation steps for this case

1. Build the service with the flags you actually ship, plus tracing:

   nim c --gc:orc --profile --threads:on --app:exe server.nim

   If you are not sure the binary uses ORC, confirm the compile flags. That changes the recommendation.
2. Enable GC summary:

   -d:gcstats

   This gives a coarse total pause metric at exit.
3. Run identical concurrent load, e.g., wrk or ab, and capture the generated nim.prof.
4. In the trace, locate intervals where orc_collect or orc_collect_all is active. Align their start/end timestamps with request latency spikes. A contiguous block of GC activity covering the spike indicates an ORC pause.
5. If the spike aligns with the asyncdispatch loop waiting on a blocking call, e.g., a synchronous I/O or user code, the spike is event-loop blocking. The stack shows the async proc and the blocking call, not the GC.
6. Control check: rebuild with --gc:refc under the same load. Disappearance of orc_collect events in the trace supports ORC attribution.

Per-thread isolation with --threads:on

With --threads:on each thread has its own asyncdispatch loop. ORC state is per thread, so a collection pause is visible only on that thread’s trace and affects only requests handled by that loop. Attribution therefore requires correlating the spike to a specific thread ID in the trace. A global spike across all threads points to a shared resource or a single-threaded deployment; isolated spikes point to per-thread GC or per-thread blocking.

Uncertainty: --profile traces can be large under high concurrency and may not capture sub-millisecond pauses accurately. -d:gcstats is exit-only and does not give per-pause timestamps; rely on the trace for fine-grained attribution. Manual collectGarbage() calls or custom GC hooks will add extra GC events to the trace.

One missing diagnostic detail that changes the recommendation: confirm the exact GC mode and thread count used in the production binary. If the binary is built with --gc:refc or without --threads:on, the ORC vs loop attribution steps above do not apply.