Scale a Node.js API on One Machine with Cluster

A single Node.js process leaves CPU cores idle

By default, a Node.js application runs on one event loop in one process. On a machine with eight CPU cores, seven cores sit unused while the active core handles every request. If that core becomes CPU-bound, response times climb even though the server has spare capacity.

The useful takeaway: Node.js includes a built-in cluster module that forks child processes, each with its own event loop, and lets them share the same TCP server port. You can use all cores on one machine without installing a reverse proxy or rewriting your application framework.

How Cluster distributes connections

The cluster module distinguishes between a master process and worker processes. The master does not handle application requests. Instead, it listens on the server port and uses the operating system to round-robin incoming TCP connections to the workers. Each worker runs the same application code independently.

Workers do not share memory. If you store session data in a global variable, that variable exists only inside the worker that created it. Sharing large objects between workers requires serialization, typically over IPC or an external store such as Redis.

Worked example: a clustered HTTP server

This example assumes Node.js 12.x or newer. Create a file named cluster-server.js:

const cluster = require('node:cluster');
const http = require('node:http');
const os = require('node:os');

const PORT = process.env.PORT || 3000;
const WORKERS = process.env.WEB_CONCURRENCY || os.cpus().length;

if (cluster.isPrimary) {
  console.log(`Master ${process.pid} is running`);

  for (let i = 0; i < WORKERS; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    if (signal) {
      console.log(`Worker ${worker.process.pid} killed by signal: ${signal}`);
    } else if (code !== 0) {
      console.log(`Worker ${worker.process.pid} exited with error code ${code}`);
    }
    console.log('Starting a replacement worker');
    cluster.fork();
  });
} else {
  http.createServer((req, res) => {
    res.writeHead(200, { 'Content-Type': 'text/plain' });
    res.end(`Handled by worker ${process.pid}\n`);
  }).listen(PORT);

  console.log(`Worker ${process.pid} started and listening on port ${PORT}`);
}

Run the file from your application directory:

node cluster-server.js

You do not need root privileges unless PORT is below 1024. The master process spawns one worker per CPU core by default. Each worker binds to the same port, and the operating system distributes incoming connections among them.

Verify the workers are running

Open a second terminal and run:

ps -ef | grep node

Expected check: you should see one master process and multiple worker processes, each with a different PID. If you see only one process, the script may have crashed or WEB_CONCURRENCY may be set to 1.

Compare throughput before and after

To observe the difference, send concurrent requests with a tool such as wrk or autocannon:

wrk -t4 -c100 -d30s http://localhost:3000/

Run the test once with WEB_CONCURRENCY=1 and once with the default core count. Compare requests per second and latency. Do not treat a single run as proof of production capacity; results vary by workload, hardware, and whether your handlers are CPU-intensive or I/O-bound.

Trade-offs and operational limits

Cluster is not a drop-in performance guarantee. Because workers are separate processes, they cannot share in-memory caches or session objects. A CPU-heavy task in one worker blocks that worker's event loop until the task finishes, although other workers can continue accepting connections.

The master process accepts connections and hands them to workers. Under extremely high connection rates, the master's accept loop can become a contention point. Monitor CPU usage and event-loop delay before adding more workers than you have cores.

Graceful shutdown matters. If you send SIGTERM to the master without handling it, workers may be killed mid-request. Add shutdown handling so workers finish active requests before exiting:

process.on('SIGTERM', () => {
  console.log('SIGTERM received: shutting down gracefully');
  server.close(() => {
    process.exit(0);
  });
});

In the clustered example above, you should also disconnect workers from the master before exiting, or rely on the master's exit handler to replace crashed workers automatically.

When to use Cluster and when to look elsewhere

Use Cluster when you run a long-lived Node.js service on a virtual machine or bare-metal host and want to use multiple cores without introducing external infrastructure. It is less suitable for serverless environments, where the runtime manages processes, or for applications that already run behind a load balancer with one Node.js process per container.

If your bottleneck is database latency or external API calls, adding workers may not improve throughput because workers spend most of their time waiting for I/O. In that case, concurrency within a single event loop may already be sufficient.

Practical closing checklist

• Confirm Node.js version with node -v; Cluster is stable in Node.js 12.x and later.
• Set WEB_CONCURRENCY to match CPU cores, or tune it based on memory usage per worker.
• Handle SIGTERM and close servers before exiting to avoid dropped requests.
• Verify with ps that multiple worker processes exist after startup.
• Load-test before and after enabling Cluster, and compare latency under realistic concurrency.

Cluster gives you multi-core use on one machine with relatively little code, but it is not a substitute for observing your actual workload. Start with one worker per core, measure, and adjust based on memory and latency rather than core count alone.