Node.js CPU‑Bound Offload: worker_threads, cluster, child_process & chunking

Problem & Decision Context

When a Node.js request path performs CPU‑bound work—cryptography, compression, image manipulation, or large‑scale data parsing—the single‑threaded JavaScript event loop stalls. In production you must keep the main thread responsive while still delivering the result. The engineering decision hinges on four constraints: shared‑memory need, fault isolation, deployment topology, and maximum acceptable main‑thread latency (SLO).

Option Comparison Table

Strategy	Shared Memory	Fault Isolation	Deployment Fit	Typical Latency Impact
worker_threads	Yes (SharedArrayBuffer + Atomics)	Shared process – crash propagates	Single process, multi‑core scaling	Low (few ms per task)
cluster	No (each worker has its own heap)	Process‑level isolation	Stateless HTTP, container autoscaling	Medium (socket‑sharing overhead)
child_process	No (IPC only)	Strong isolation, safe for native segfaults	Any service, especially untrusted binaries	High (serialization & process start)
Cooperative chunking	No	None – runs in main thread	Workloads naturally <10 ms	Very low (no context switch)

Trade‑off Deep Dive

worker_threads

Best when you need high‑throughput numeric or buffer work and can tolerate the shared‑process crash risk. Workers share the same V8 heap, so a crash kills the whole process. Use a pool with health checks and timeouts to mitigate this. Transferable ArrayBuffers avoid copying overhead.

cluster

Lowest friction for HTTP servers that are already stateless. The primary process forks workers that all listen on the same port. Sticky sessions and in‑process caches become more complex because each worker has its own memory. Ideal when you can afford a duplicate heap per core.

child_process

Use when you run heavy native modules or third‑party binaries that might segfault. The IPC channel serializes JavaScript objects; for large buffers you must use send with transferList. The startup cost is higher, but isolation is strongest.

Cooperative chunking

If the CPU work can be broken into <10 ms slices, you can keep the main thread responsive without spawning extra processes. Use setImmediate or scheduler.yield() after each slice. This is the simplest but only helps tasks that naturally fit into small chunks.

Concrete Implementation: Worker‑Thread Pool

The following minimal pool offloads a CPU‑heavy function (e.g., SHA‑256 hashing) to worker_threads. Workers are reused, timed‑out jobs are terminated, and crashed workers are respawned automatically.

/* cpuPool.js */
const { Worker } = require('worker_threads');
const path = require('path');

const MAX_WORKERS = 4; // tune per host
const JOB_TIMEOUT_MS = 5000;

class WorkerPool {
  constructor() {
    this.workers = [];
    this.idle = [];
    this.taskQueue = [];
    this.taskMap = new Map();
    for (let i = 0; i < MAX_WORKERS; i++) this._addWorker();
  }

  _addWorker() {
    const worker = new Worker(path.resolve(__dirname, 'worker.js'));
    worker.on('message', msg => this._handleResult(worker, msg));
    worker.on('error', err => this._handleError(worker, err));
    worker.on('exit', code => this._handleExit(worker, code));
    this.workers.push(worker);
    this.idle.push(worker);
  }

  _handleResult(worker, msg) {
    const { id, result } = msg;
    const job = this.taskMap.get(id);
    if (job) {
      clearTimeout(job.timeout);
      job.resolve(result);
      this.taskMap.delete(id);
    }
    this.idle.push(worker);
    this._processQueue();
  }

  _handleError(worker, err) {
    console.error('Worker error:', err);
    this._replaceWorker(worker);
  }

  _handleExit(worker, code) {
    if (code !== 0) console.warn('Worker exited abnormally', code);
    this._replaceWorker(worker);
  }

  _replaceWorker(worker) {
    this.workers = this.workers.filter(w => w !== worker);
    this.idle = this.idle.filter(w => w !== worker);
    this._addWorker();
  }

  _processQueue() {
    if (!this.taskQueue.length || !this.idle.length) return;
    const { job, resolve, reject } = this.taskQueue.shift();
    const worker = this.idle.shift();
    const id = Symbol();
    const timeout = setTimeout(() => {
      reject(new Error('Worker timeout'));
      worker.terminate();
      this._replaceWorker(worker);
    }, JOB_TIMEOUT_MS);
    this.taskMap.set(id, { resolve, reject, timeout });
    worker.postMessage({ id, job });
  }

  run(job) {
    return new Promise((resolve, reject) => {
      this.taskQueue.push({ job, resolve, reject });
      this._processQueue();
    });
  }
}

module.exports = new WorkerPool();

/* worker.js */
const { parentPort } = require('worker_threads');
const crypto = require('crypto');

parentPort.on('message', ({ id, job }) => {
  const hash = crypto.createHash('sha256');
  hash.update(job.data);
  const digest = hash.digest();
  parentPort.postMessage({ id, result: digest });
});

Usage in an HTTP handler:

const pool = require('./cpuPool');

app.post('/hash', async (req, res) => {
  try {
    const result = await pool.run({ data: req.body.buffer });
    res.send(result);
  } catch (err) {
    res.status(500).send(err.message);
  }
});

Validation & Monitoring

Measure the event‑loop delay before and after enabling the pool:

const { monitorEventLoopDelay } = require('perf_hooks');
const h = monitorEventLoopDelay({ resolution: 10 });
h.enable();
// ... run load test ...
console.log('p99 delay:', h.percentile(99));

Run a mixed load test with autocannon targeting a CPU endpoint while another endpoint performs I/O. Verify that the p99 event‑loop delay stays below the SLO (e.g., <10 ms). Also instrument the pool to emit worker-crashed events and assert that the pool recycles workers within a timeout.

Version Sensitivity & Pinning

All features used (worker_threads, perf_hooks.monitorEventLoopDelay, Atomics.waitAsync) are stable on Node 18+ LTS. Pin the runtime with an .nvmrc and engines field, then re‑run the latency harness on every Node upgrade. Verify that SharedArrayBuffer is available on the target platform before using it.

Practical Checklist

• Does the workload require shared state? If yes, use worker_threads with SharedArrayBuffer.
• Is the service stateless and deployed in a container cluster? Prefer cluster.
• Do you call potentially unsafe native code? Use child_process for isolation.
• Can the work be broken into <10 ms slices? Use cooperative chunking.
• Monitor EventLoopDelay and set a hard timeout on worker tasks.
• Recreate the pool on worker crash and limit per‑worker memory.