Dyalog Isolates for Parallel Array Work Without Shared Memory

The problem is one core, not one array

A CPU-bound APL expression like a large matrix factorisation or a per-row simulation can saturate a single interpreter. Dyalog primitives are implicitly parallel, but one workspace is one OS process. The engineering question is how to use more cores without introducing locks, threads, or manual chunking of shared state.

What an isolate actually is

An isolate is a separate Dyalog APL process. It has its own workspace, its own ⎕WA, and its own execution stack. The parent talks to it through the isolate namespace and the system functions ⎕NS, ⎕CS and ⎕SH.

⎕NS is used to evaluate namespaced code inside the isolate. ⎕CS sends a command string and returns the result. ⎕SH shuts the isolate down. Messages are APL values serialized by the interpreter, typically as native binary or via ⎕FMT/⎕XML for interoperability. Because serialization is involved, large arrays incur copy cost.

Isolates can run locally on the same machine or on remote hosts over TCP. Remote use requires matching Dyalog versions and network access to the target port. The isolate inherits the parent workspace only at creation time; later code changes in the parent are not visible unless the isolate is restarted.

A minimal spawn-send-collect pattern

Run the following in a Dyalog session where isolate support is loaded. Check availability first with ]Isolates in the session console. No elevated permissions are required for local isolates; remote isolates require network reachability to the target host.

⎕IO←0
n←4
isos←n⍴⎕NEW 'isolate'

This creates n independent interpreters. Each isolate starts empty. You can seed common code once per isolate with ⎕NS.

work←{⍵*2}
⊃isos ⎕NS 'f←{⍵*2}'

Send a work package and collect. Use the vector of numbers 1..n as input.

inputs←⍳n
results←isos ⎕CS 'f'¨inputs

Expected checks: each isolate returns the square of its input. The parent workspace remains unchanged. Unhandled errors inside an isolate are trapped with ⎕TRAP in the isolate; they terminate that isolate but do not terminate the parent.

Shutdown when done to release processes.

⊃isos ⎕SH

Risk: forgetting to shut down leaves orphan Dyalog processes consuming memory and CPU.

Trade-offs and when not to use isolates

Startup cost is non-trivial. Isolates are unsuitable for fine-grained parallelism where work units complete in microseconds. Batch work units that run for tens of milliseconds or more amortise the spawn and serialization overhead.

Serialization overhead dominates for very large arrays. If the data does not fit comfortably in memory twice, consider memory-mapped files or component files accessed by each isolate instead of sending the array by message.

Error handling is per process. A crash in one isolate does not crash the parent, but the parent must detect missing results and decide whether to retry or replace the isolate.

Version assumptions matter. Isolate behaviour and the ⎕NS/⎕CS API are version specific. Verify with Help → Isolates in the Dyalog IDE and benchmark with ⎕AI to compare single-threaded versus isolate execution for your workload.

Check it works

A quick sanity test without data movement:

iso←⎕NEW 'isolate'
iso.⎕EX 'x'
iso.x←42
iso.x

Check that the isolate namespace responds and that the value round-trips. Then measure wall time for a representative chunk size with ⎕AI and compare to a single workspace run. If the isolate run is slower, the work units are too small or serialization is too high.

Isolates give you process-level parallelism with APL-native messaging. Use them for coarse-grained, CPU-bound array tasks where isolation and simplicity outweigh the cost of serialization.