Erlang OTP Supervisor Crash Loops: Diagnosis and Recovery Guide

The problem you can recognize in logs

A supervisor repeatedly reports that the same child died and then restarts it, until the supervisor itself terminates with "reached maximum restart intensity". The child never reaches a stable state. The useful takeaway is that the supervisor is working as designed; the crash loop is a symptom of a child that cannot start cleanly, not a supervisor bug.

Restart intensity is defined by two values: MaxR and MaxT. The supervisor allows at most MaxR restarts within a sliding window of MaxT seconds. When the limit is exceeded the supervisor exits with reason shutdown and propagates the failure upward.

Symptom to cause mapping

Ordered checks

1. Read the supervisor and child exit reason

From an Erlang shell connected to the running node, inspect the supervisor state. You need operator access to the node.

erlang:display(supervisor:which_children(SupervisorName)).

Check the application logs for the supervisor report block. The report includes the child id, the exit reason, and the restart count within the intensity window. Do not change intensity yet.

2. Verify child start arguments and environment

Find the child spec in the supervisor module. A typical one_for_one spec looks like:

{ChildId, {Mod, start_link, [Arg1, Arg2]}, Restart, Shutdown, Type, Modules}

Confirm that Arg1, Arg2 match what Mod:start_link/1 expects. Check application environment values used to build the args. A mismatch here causes immediate exit with badarg or function_clause.

3. Test the child in isolation

Start the child module manually in a clean shell on the same node. Run it with the same arguments the supervisor passes. Required permission: shell access to the node. Risk: manual start bypasses supervision, so clean up the process afterwards.

{ok, Pid} = my_child:start_link(Args).

If it crashes here, the problem is in the child, not the supervisor.

4. Check system resource limits

Run on the host OS as an operator. Resource exhaustion often appears as exit reason {system_limit,_} or as a timeout after a few seconds.

ulimit -n

Compare current open file descriptors to the limit. Also check the Erlang VM port limit and max processes. If limits are near exhaustion, the child cannot open files or sockets during init.

5. Review supervisor strategy and intensity

Inspect the supervisor init return:

{ok, {{Strategy, MaxR, MaxT}, [ChildSpecs]}}

Strategy one_for_one restarts only the failed child. one_for_all and rest_for_one restart groups. Changing strategy without understanding dependencies can cause cascading failures.

Fixes tied to findings

Bad arguments or environment

Correct the child spec Args and the application environment. Recompile and restart the application. Rollback is redeploying the previous release if the child still fails to start.

Unhandled exception in init

Ensure init returns {ok, State} or {ok, State, Timeout}. Do not let exceptions propagate. Wrap risky startup code with try...catch and return a clean error tuple so the supervisor can log a meaningful reason.

Resource exhaustion

Close file handles and ports in terminate/2. Remove leaks in handle_info or handle_call. Raise OS limits only after confirming the leak is fixed. Increasing ulimit without fixing the leak masks the bug and risks exhaustion of the whole node.

Dependency ordering

If the child needs another service, either start the dependency in the same supervisor with a lower id, or use a separate supervisor with a temporary restart strategy and a delayed retry. Do not increase MaxR to hide a dependency race.

Intensity tuning

Adjust MaxR and MaxT only after the root cause is fixed. A temporary increase can be used to avoid immediate escalation during a controlled rollout, but it should be reverted.

Escalation criteria

Escalate to the parent supervisor when:

• The supervisor has exceeded MaxR restarts in MaxT seconds and terminates.
• The child requires a manual intervention such as a data fix or external service restoration.
• Multiple children in a one_for_all supervisor fail repeatedly, indicating a shared dependency failure.

When escalation occurs, the parent supervisor will see the child supervisor exit with reason shutdown. The application should be designed so that a higher-level supervisor can restart the whole subsystem cleanly.

Limitations and verification

This guide assumes OTP supervisor behavior where restart intensity is enforced per supervisor. It does not cover distributed node partitions or net_kernel issues.

Verify the fix by confirming the child reaches a stable state and the supervisor restart count stays below MaxR. Use observer to watch the supervisor tree and confirm no repeated restarts. Trace supervisor events with dbg only in a staging environment, as tracing adds overhead.

Avoid simply increasing max_restart intensity without fixing the child process. That masks underlying bugs and can lead to resource exhaustion.