Designing a PM2‑clustered Node.js API: Requirements, Minimal Config, Boundaries, Checks, and Failure Modes

Problem and Takeaway

You need to run a Node.js API service on multiple CPU cores without rewriting the application for clustering. The goal is to let PM2 handle process replication while keeping the service stateless, observable, and resilient. The takeaway is a minimal, production‑ready design that defines the exact configuration, trust boundaries, operational checks, and the conditions that would force you to change the architecture.

Requirements

Before adding PM2 cluster mode, verify that the service meets these prerequisites:

1. Statelessness or externalized state: any session data, cache, or shared state must live outside the Node.js process (e.g., Redis, a database, or an external object store).
2. Health/readiness endpoint: an HTTP route such as /health that returns 200 when the worker is ready to serve traffic.
3. Predictable resource usage: the app’s CPU and memory footprint should be roughly uniform across instances so PM2 can decide how many workers to spawn based on available cores.
4. Graceful shutdown handling: the process must listen for SIGINT and SIGTERM, finish in‑flight requests, and exit with a zero status code.

If any of these are missing, cluster mode will either produce incorrect behavior or hide failures.

Smallest Suitable Design

The design consists of three files: an ecosystem configuration, a systemd service unit, and (optional) a logrotate configuration that PM2 manages internally.

ecosystem.config.js

Place this file in the project root. It tells PM2 how to launch the workers.

module.exports = {
  apps : [
    {
      name      : 'api-service',
      script    : './src/index.js',
      instances : 'max',          // use all available CPU cores
      exec_mode : 'cluster',      // enable cluster mode
      watch     : false,          // disable file watching in production
      max_memory_restart : '200M',// restart if a worker exceeds this memory
      env       : {
        NODE_ENV: 'production',
        PORT    : 3000
      },
      log_date_format : 'YYYY-MM-DD HH:mm:ss.Z'
    }
  ]
};

Where to run: on the host where the service will execute, as a non‑root user (see systemd unit). Required permissions: read access to the application files and write access to the PM2 home directory (~/.pm2). Expected check: after starting, pm2 list shows online instances equal to the number of CPU cores (nproc). Risk: setting instances to a fixed number higher than the core count can cause oversubscription and increased context‑switch overhead.

systemd Service Unit

Create /etc/systemd/system/api-pm2.service:

[Unit]
Description=PM2 process manager for api-service
After=network.target

[Service]
User=pm2user               # dedicated non‑root user
Group=pm2user
Environment=PATH=/usr/bin:/usr/local/bin
Environment=PM2_HOME=/home/pm2user/.pm2
ExecStart=/usr/local/bin/pm2 start /opt/api-service/ecosystem.config.js
ExecStop=/usr/local/bin/pm2 dump && /usr/local/bin/pm2 kill
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target

Where to run: on the host with root privileges to create the unit file; the service itself runs as pm2user. Required permissions: root to write to /etc/systemd/system and to enable the unit; the pm2user needs read/execute on the application directory and write on its home. Expected check: systemctl status api-pm2 shows active (running) and pm2 jlist lists the expected number of workers. Risk: if the unit file incorrectly references a non‑existent user, the service will fail to start; verify the user exists before reloading systemd.

Log Rotation (PM2 built‑in)

PM2 can rotate logs without external tools. Add to ecosystem.config.js under the app definition:

error_file : './logs/err.log',
out_file   : './logs/out.log',
merge_logs : true,
log_date_format : 'YYYY-MM-DD HH:mm:ss.Z',
max_size   : '10M',          // rotate after 10 MB
max_files  : '5',            // keep 5 rotated files

Expected check: after a few minutes of traffic, ls -lh ./logs shows rotated files like out.log.1 and the current out.log stays under the size limit. Risk: disabling rotation or setting an excessively large max_size can fill the disk.

Trust and Data Boundaries

PM2 runs as the OS user that starts it (here, pm2user). Each worker is a separate forked Node.js process with its own V8 heap, so there is no intra‑process shared memory. The OS process boundary is therefore the trust boundary: PM2 is trusted to manage processes, while the workers execute the application code, which must be considered untrusted from a data‑isolation perspective. Any data that needs to be shared across workers must travel through an external store (Redis, database, or a message queue). This keeps the security model simple: compromise of one worker does not automatically give access to another’s memory.

Operational Checks

To verify that the design works in practice, perform these checks regularly:

• Instance count: pm2 list → online matches nproc (or the fixed number you set).
• Health endpoint: curl each worker individually (e.g., curl -s http://localhost:3000/health) or via a load balancer; all should return 200.
• Metrics: enable PM2’s built‑in monit (pm2 monit) or export metrics to Prometheus/Datadog; watch for steady CPU (< 80 % per core) and memory below the max_memory_restart threshold.
• Log rotation: confirm that pm2 logs shows recent output and that rotated files appear in the log directory with the configured retention.
• Daemon persistence: systemctl is-enabled api-pm2 returns enabled and a reboot results in the same instance count after startup.
• Zombie processes: after a restart, ps -ef | grep node shows only the expected workers and no defunct (Z) processes.

Where to run: on the host, as the pm2user (for PM2 commands) or with sudo for systemd checks. Required permissions: read access to PM2 logs and the ability to query the process table. Expected checks: all metrics within thresholds, no unexpected restarts. Risk: if the health endpoint is slow or fails, the load balancer may mark workers as unhealthy, causing traffic loss.

Failure Modes and Design‑Change Triggers

Certain conditions invalidate the assumptions of the minimal design and would require you to change the architecture:

1. In‑process shared state: if the application uses local variables, Map, or sticky sessions, cluster mode will cause data inconsistency. Trigger: observe divergent responses for the same request sent to different workers. Remedy: externalize the state (Redis, database) or disable cluster mode (instances: 1) and scale horizontally with a process manager like Docker Swarm or Kubernetes.
2. Long startup time: if the worker needs several seconds to initialize, a graceful reload (pm2 reload) will cause a brief downtime as each worker restarts sequentially. Trigger: monitoring shows increased error rates during pm2 reload. Remedy: implement a zero‑downtime strategy using PM2’s --no-daemon with a blue‑green deployment or use a reverse proxy that can drain connections before stopping a worker.
3. Windows support: PM2’s cluster mode relies on fork, which is limited on Windows. Trigger: deployment to a Windows host fails to start workers or shows only one instance. Remedy: use PM2’s fork mode (exec_mode: 'fork') or run the service inside a Linux container.
4. Strict sandboxing: security policies that require each worker to run under a different user or container cannot be satisfied by a single PM2 daemon. Trigger: audit reveals that all workers share the same UID, violating policy. Remedy: run multiple PM2 daemons, each configured for a specific user, or orchestrate with Kubernetes where each pod gets its own security context.
5. Resource spikes: if memory usage varies wildly between workers, the fixed max_memory_restart may cause premature restarts or, conversely, allow a leak to grow unchecked. Trigger: frequent restarts visible in pm2 logs with "memory limit exceeded". Remedy: tune the threshold based on observed workload, or switch to a container‑orchestrated setup where resource limits are enforced by the runtime.

Rollback Considerations

Changing the ecosystem.config.js file or the systemd unit does alter system state (the running processes). To roll back:

1. Edit the file to the previous known‑good version.
2. Run sudo systemctl daemon-reload to refresh systemd.
3. Restart the service: sudo systemctl restart api-pm2.
4. Verify with pm2 list that the instance count matches the rolled‑back configuration.

If the rollback fails to start, check journalctl -u api-pm2 for error messages and ensure the file paths and permissions are correct.