Using Dyalog APL’s .PARALLEL Directive to Parallelize Matrix Multiplication

The problem

Matrix multiplication is a core operation in many workloads. In Dyalog APL the expression A +.× B is concise, but as the matrices grow the single‑threaded evaluation becomes a bottleneck. Rewriting the algorithm with explicit parallel loops sacrifices APL’s readability and introduces synchronization concerns.

What .PARALLEL does

The .PARALLEL directive, available from Dyalog 18.0, is a compiler hint that tells the interpreter to distribute the leading axis of a primitive across the available CPU cores. For +.× the leading axis corresponds to the rows of the left argument; each worker computes a contiguous block of result rows. The runtime handles partitioning, scheduling, and assembling the final matrix, so the code still looks sequential.

To enable parallelism you place the directive immediately before the function:

      .PARALLEL A +.× B

The interpreter creates a thread pool (default size = number of logical cores), splits the row range into chunks, and dispatches each chunk to a worker. Because APL arrays are immutable, workers can share the input data without copying unless a write‑back copy is needed.

Worked example

The following session shows how to apply the directive and measure elapsed time with the built‑in timer ⎕AI. Adjust the matrix size to suit your hardware.

      ⎕← 'Building test matrices…'
      A ← 5000 5000 ⍴ 1e6 ?@⍴ 0   ⍝ random single‑precision floats
      B ← 5000 5000 ⍴ 1e6 ?@⍴ 0
      ⎕← 'Sequential multiplication…'
      t1 ← ⎕AI[3] ⋄ C1 ← A +.× B ⋄ dt1 ← (⎕AI[3]-t1)÷1000
      ⎕← 'Parallel multiplication…'
      t2 ← ⎕AI[3] ⋄ C2 ← .PARALLEL A +.× B ⋄ dt2 ← (⎕AI[3]-t2)÷1000
      ⎕← ('Sequential time (s):' dt1) ('Parallel time (s):' dt2)
      ⎕← ('Result equality:' C1 ≡ C2)

No specific output is shown here; you should observe that dt2 is smaller than dt1 for sufficiently large matrices, and that the equality test returns 1 indicating bitwise‑identical results.

Verifying parallel execution

1. While the parallel expression runs, open a system monitor (Task Manager on Windows, htop on Linux, Activity Monitor on macOS). You should see elevated usage on multiple logical cores.
2. Compare the wall‑clock time from ⎕AI[3] (or ⎕TS) with the CPU time reported by the OS; the wall time should drop while the total CPU time stays roughly constant.
3. In Dyalog 18.2+ you can run ]runtime -c to inspect thread‑pool statistics.

Trade‑offs and limits

• Startup overhead: Creating the thread pool and dispatching chunks adds a few‑tens of milliseconds. For matrices smaller than about 1000×1000 the parallel version may be slower than the sequential one.
• Memory consumption: Each worker holds a view of the input arrays. If the operation forces a copy (e.g., when writing into a pre‑allocated buffer), memory use can rise with the core count. Monitor workspace with ⎕WA or OS tools.
• Non‑uniform workloads: For dense +.× the rows have similar cost, but if you apply .PARALLEL to a primitive with highly variable per‑row work, static partitioning could leave some cores idle. Newer Dyalog releases add dynamic scheduling for certain primitives; consult the release notes.
• Floating‑point reproducibility: The order of additions differs across partitions, so the least significant bit of the result may vary between runs. If bitwise‑identical output is required, use decimal floating‑point (⎕FR←1287) or accept the small variance.

When to use .PARALLEL

Apply the directive to dense linear‑algebra kernels (matrix multiply, ⌹ for solve, outer products) when the matrices are large enough that the parallel speedup outweighs the fixed overhead. Start with a benchmark at your target size; if the wall‑clock time drops noticeably and memory headroom remains comfortable, keep the directive in production code. For small arrays or memory‑bound operations, the sequential version is preferable.