Nim --gc:refc cycle collector pause latency under concurrent request threads

Factors that determine pause frequency

The refc cycle collector runs when the number of unreachable cyclic objects reaches an internal threshold. This threshold depends on the allocation rate of cyclic structures, the current size of the heap, and the frequency with which mutator threads acquire the global heap lock.

Confirmed facts

• Each cycle collection triggers a stop‑the‑world (STW) pause that suspends all mutator threads.
• The pause duration grows with the number of live objects that must be scanned to find cycles.
• Under high thread contention, the global heap lock can cause additional blocking even before the STW phase begins.

Likely explanation

When request concurrency exceeds the capacity of a single event loop, more threads allocate cyclic objects simultaneously, pushing the allocation rate higher and reaching the threshold more often. The markAndSweep collector, by contrast, performs a full heap traversal less frequently but each pause scales with the total live heap size, leading to longer but less frequent STW events.

Pause‑bound behavior comparison

GC backend	Typical pause pattern	Latency impact under high concurrency
--gc:refc	Frequent tiny refcount updates; periodic STW for cycle collection	Lower average latency but higher tail variance due to unpredictable cycle‑trigger spikes
--gc:markAndSweep	Infrequent full‑heap STW pauses	Higher baseline latency that grows with heap size; tail less variable but pauses can be longer

Why deterministic latency guarantees are absent

Nim's GC backends are specified as implementation‑defined; they target throughput and memory safety rather than hard real‑time bounds. Pause timing depends on the OS scheduler, heap fragmentation, and the internal threshold that is not exposed as a stable API, so no version‑independent guarantee can be given.

Verification steps

1. Compile the service with tracing: nim c --gc:refc --trace:gc.nim app.nim
2. Run a realistic load generator (e.g., wrk -t8 -c100 -d30s "http://localhost:8080") while recording timestamps.
3. Examine the trace output for lines like GC: cycle collection start and GC: cycle collection end to measure pause duration.
4. If the observed tail latency does not match expectations, collect the allocation rate of cyclic objects (e.g., via a custom counter) to see whether the threshold is being hit frequently.

One missing diagnostic detail that would change the recommendation is the measured allocation rate of cyclic structures per second; knowing this lets you compare it to the internal threshold and decide whether to increase the threshold via a compiler pragma or switch GC backend.