Diagnosing PM2 Cluster Mode Misbehavior: An Engineering Decision Guide

Recognizing the Problem

When a Node.js application is run under PM2 in cluster mode, you may notice one or more of the following symptoms:

• Uneven request distribution – some workers handle far more traffic than others.
• Frequent worker crashes or restarts triggered by max_memory_restart.
• CPU under‑utilization or a single worker hogging all CPU cycles.
• Workers stuck in a stalled state, event loop blocked by synchronous code.
• Cluster mode appears disabled even though you configured it.

These issues often stem from misconfigurations, environment constraints, or application code patterns that conflict with PM2’s clustering model.

Diagnostic Table

Symptom	Possible Cause	Primary Diagnostic Check
Uneven traffic	Sticky sessions or load‑balancer mis‑config	Verify load‑balancer algorithm
Frequent restarts	Memory leak or low max_memory_restart	Check PM2 logs for memory warnings
CPU imbalance	Single worker handling all requests	Inspect pm2 monit CPU per worker
Stalled workers	Blocking synchronous code	Search logs for event loop lag warnings
Cluster disabled	--no-cluster flag or env override	Review ecosystem file and env vars

Step‑by‑Step Checks

1. Confirm Cluster Mode is Enabled

   pm2 show <app-name>

   Look for mode: cluster in the output. If it shows mode: fork, cluster is disabled.

2. Validate Instances Setting

   pm2 ecosystem

   Open ecosystem.config.js and ensure instances: 'max' or a numeric value is set. The default is 1, which disables clustering.

3. Inspect Load Balancer Configuration
   • If you are using an external LB (NGINX, HAProxy, AWS ELB), check that it uses round‑robin or consistent hashing rather than sticky sessions.
   • In the application, ensure no session‑affinity middleware forces requests to the same worker.
4. Monitor CPU and Memory per Worker

   pm2 monit

   Verify that each worker shows similar CPU usage. If one worker dominates, it may be handling all traffic.

5. Check for Memory‑Restart Events

   pm2 logs <app-name> | grep "max_memory_restart"

   Frequent entries indicate the threshold is too low or the app leaks memory.

6. Detect Event Loop Lag

   pm2 logs <app-name> | grep "event loop lag"

   Large lag values point to blocking synchronous operations.

7. Validate Environment Flags

   echo $NODE_OPTIONS

   Ensure --no-cluster is not set inadvertently.

Fixes Tied to Findings

• Uneven Traffic
  • Reconfigure the load balancer to use round‑robin or consistent hashing.
  • Remove sticky‑session middleware or set app.set('trust proxy', 1) and use express-session with a shared store.
• Frequent Restarts
  • Increase max_memory_restart in ecosystem.config.js:

    module.exports = {
      apps: [{
        name: 'myapp',
        script: 'server.js',
        instances: 'max',
        max_memory_restart: '1G',
        env: { NODE_ENV: 'production' }
      }]
    };

  • Profile the application with clinic.js or node --inspect to locate memory leaks.
• CPU Imbalance
  • If a single worker is hogging CPU, verify that the worker count matches the number of CPU cores: instances: 'max' will spawn os.cpus().length workers.
  • Check for heavy synchronous code in request handlers. Refactor to async/await or offload to worker threads.
• Stalled Workers
  • Search the codebase for long synchronous loops, fs.readFileSync, or blocking crypto calls.
  • Replace with async equivalents or move heavy tasks to worker_threads or a dedicated microservice.
• Cluster Disabled
  • Remove --no-cluster from node_args or NODE_OPTIONS.
  • Set cluster: true in the ecosystem file if you use the pm2 start command with the --cluster flag.

Escalation Criteria

If after applying the above fixes the problem persists, consider the following:

• Run a controlled load test (e.g., wrk -t12 -c400 -d30s http://localhost:3000/) and capture worker request counts with pm2 logrotate or custom instrumentation.
• Review Node.js version compatibility – cluster mode is stable on Node 12+, but older releases may have IPC bugs.
• Examine shared global state: if workers modify shared objects, race conditions can cause erratic behavior. Migrate to a Redis or database store.
• If the application uses native addons, ensure they are thread‑safe and compiled for the target architecture.
• Contact PM2 support or file an issue on the PM2 GitHub repository with detailed logs and environment info.

Practical Verification Checklist

• Run pm2 list – all instances should show online status.
• Confirm pm2 show <app> displays mode: cluster and the correct number of instances.
• Use pm2 monit to see balanced CPU/memory across workers.
• Check logs for absence of max_memory_restart or event loop lag entries after load test.
• If using a load balancer, run a traffic distribution test (e.g., ab -n 10000 -c 100 http://lb/) and confirm even request counts per backend IP.