PureScript Aff fiber scheduling and event loop latency
0 reputation · 21 Nov 2022, 20:28 UTC
0 reputation · 21 Nov 2022, 20:28 UTC
PureScript manages asynchronous operations through the Aff monad, which leverages the JavaScript event loop to handle non-blocking I/O. While primitives like parallel and race allow multiple computations to initiate concurrently, the underlying execution remains single-threaded.
When deploying a high volume of concurrent fibers using fork, the interaction between the PureScript runtime and the JS engine's task queue can introduce variable latency. Specifically, there is uncertainty regarding how the runtime prioritizes fiber resumption when multiple Aff computations resolve nearly simultaneously.
If a system relies on heavy use of Ref for shared state across these fibers, the lack of OS-level parallelism means synchronization is handled via the event loop, but the specific ordering of fiber wake-ups is not explicitly guaranteed.
purescript-aff scheduler implement a specific priority queue for fiber resumption?33275 reputation · 21 Nov 2022, 22:34 UTC
No, the purescript-aff scheduler does not maintain a priority queue for fiber resumption. Fibers are resumed in roughly the order their continuations become runnable, and that order is ultimately decided by the JavaScript engine's microtask/macrotask queues, not by Aff. And Aff does not actively prevent event-loop starvation: a fiber that runs a long synchronous stretch never yields, so it monopolizes the single thread exactly like any other long-running JavaScript function. Starvation avoidance is your responsibility, via explicit yields.
Aff compiles each fiber into a chain of small continuations. When a fiber is forked or an async result arrives, the continuation is handed to the JavaScript runtime (via resolved promises / microtasks, and timer callbacks for delay). The scheduler then runs a fiber's queued steps until it hits an asynchronous boundary, a fork/join interaction, or an explicit yield, at which point control returns to the event loop and the next pending task runs.
Two consequences follow directly:
Ref updates are safe from data races precisely because only one fiber executes at a time — but only between yield points. Treat every Aff bind that can suspend as a potential interleaving point; code between two yields is effectively atomic.The common cause is not fiber count but uninterrupted work: a tight loop (parsing, folding a large array, encoding JSON) inside one fiber runs to completion before any other fiber or browser task gets a turn. Thousands of idle fibers are cheap; one compute-bound fiber is what stalls the loop.
delay (Milliseconds 0.0) (or an equivalent explicit yield) so pending fibers and I/O callbacks run.AVars or a pooled fork) instead of forking unboundedly, so wake-up bursts stay small.chunked :: forall a. Array a -> (a -> Aff Unit) -> Aff Unit
chunked xs f = go 0
where
go i = case xs !! i of
Nothing -> pure unit
Just x -> do
f x
when (i `mod` 50 == 0) (delay (Milliseconds 0.0))
go (i + 1)
The internals above describe long-standing behavior, but the scheduler is not a stable public API and details may shift between purescript-aff releases — check the source of the version you depend on. To confirm the latency story in your environment, measure the event loop directly: in Node, sample performance.now() on a repeating setImmediate and compare max gap with and without yields in your hot loop; in a browser, watch the same via a requestAnimationFrame delta or the Performance panel. If your spikes persist after chunking, the missing diagnostic is whether the blocking work is actually in Aff at all — a native binding (JSON.stringify, a FFI call) won't yield no matter how you structure the Aff code around it.
Use comments to ask for clarification. Post a solution as an answer.
33,275 reputation · 21 Nov 2022, 21:36 UTC
One point worth adding to the answer above: the safety of Ref under Aff is frequently overstated. Single-threaded execution protects you from torn reads and writes, but not from interleaving bugs — any suspending bind between a Ref.read and a Ref.write is a window where another fiber can mutate the same cell, so read-modify-update sequences are not atomic unless they contain no async boundaries.
For the coordination patterns the question hints at (handoff, gating, backpressure), Aff.AVar is the better primitive: AVar.take suspends a fiber until a value is produced, giving you blocking semantics without burning the event loop or depending on wake-up order. That sidesteps the unspecified resumption ordering entirely, because correctness no longer depends on which fiber runs first.
For the starvation side, a practical check: spawn one fiber running a tight synchronous loop alongside timer-based fibers and log their completion times. If the timers slip, insert an explicit yield (e.g., a zero-delay delay or the yield primitive your purescript-aff version exposes) inside the loop. Behavior here is version-sensitive, so verify against the scheduler source in your installed package set rather than assuming.