Designing a Minimal OTP Supervision Tree for Fault‑Tolerant Erlang Services

Requirements

The service must isolate failures so that a crash in one component does not bring down others, restart failed components automatically, and keep a clear message contract between supervised workers and their supervisors.

Smallest Suitable Design

One‑for‑one supervisor wrapping a single gen_server worker. The supervisor lives in its own module (myapp_sup) and is started as part of an OTP application (myapp).

Trust and Data Boundaries

• Workers trust only messages that match the defined protocol (e.g., {request, Payload} or {reply, Result}). Any unexpected term is treated as a programmer error and should cause a crash.
• Supervisors trust only the exit reasons reported by linked child processes. They interpret normal shutdown (normal, shutdown) as expected; any other reason (kill, error) is considered a failure and triggers a restart according to the restart strategy.

Operational Checks

1. Supervisor logs start/stop events via error_logger:info_msg/2 (or logger in OTP 25+).
2. Heartbeat or status metrics can be scraped with observer (observer:start/0) or exported through telemetry.
3. A simple CLI command (myappctl status) can call supervisor:which_children(myapp_sup) and print the list.

Failure Modes and Design Triggers

If the worker exits with a reason that is not considered normal more than MaxR times within MaxT seconds, the supervisor reaches its maximum restart intensity and shuts down itself. This propagates to the application controller, which stops the whole application.

Conditions that would force a redesign:

• Frequent restarts indicate the worker is not robust enough – increase internal error handling or make the worker stateless.
• Isolation is insufficient; add a higher‑level supervisor that groups several workers.
• The worker holds non‑recoverable state; consider persisting state or using a gen_statem with recovery callbacks.

Example Implementation

%% myapp_sup.erl
-module(myapp_sup).
-behaviour(supervisor).

%% API
-export([start_link/0]).

%% Supervisor callbacks
-export([init/1]).

start_link() ->
    supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
    RestartStrategy = {one_for_one, 5, 10}, % 5 restarts in 10 seconds
    ChildSpec = #{id => myapp_worker,
                  start => {myapp_worker, start_link, []},
                  restart => permanent,
                  shutdown => 5000,
                  type => worker,
                  modules => [myapp_worker]},
    {ok, {RestartStrategy, [ChildSpec]}}.

%% myapp_worker.erl
-module(myapp_worker).
-behaviour(gen_server).

%% API
-export([start_link/0, request/1]).

%% gen_server callbacks
-export([init/1, handle_call/3, handle_cast/2, handle_info/2,
         terminate/2, code_change/3]).

start_link() ->
    gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

request(Payload) ->
    gen_server:call(?MODULE, {request, Payload}).

init([]) ->
    % initialize any state here
    {ok, #state{}}.

handle_call({request, Payload}, _From, State) ->
    % process payload, may crash on unexpected input
    Result = process(Payload),
    {reply, Result, State};
handle_call(_Request, _From, State) ->
    {reply, unexpected, State}.

handle_cast(_Msg, State) ->
    {noreply, State}.

handle_info(_Info, State) ->
    {noreply, State}.

terminate(_Reason, _State) ->
    ok.

code_change(_OldVsn, State, _Extra) ->
    {ok, State}.

process(Payload) when is_list(Payload) ->
    % example: just return length
    length(Payload);
process(_) ->
    % unexpected term -> crash
    erlang:error(badarg).

%% myapp.app
{application, myapp,
 [
  {description, "Example OTP application with a supervision tree"},
  {vsn, "1.0.0"},
  {modules, [myapp_sup, myapp_worker]},
  {registered, [myapp_sup, myapp_worker]},
  {applications, [kernel, stdlib]},
  {mod, {myapp_app, []}},
  {env, []}
 ]}.

Verification Steps

1. Compile the modules: erlc myapp_sup.erl myapp_worker.erl.
2. Start the application: erl -pa . -s myapp start (requires read/write permission in the current directory).
3. Check the console for an info log similar to INFO SUPERVISOR myapp_sup started.
4. Launch observer:start/0 from the Erlang shell; you should see myapp_sup with a child myapp_worker under it.
5. To simulate a crash, call from the shell: rpc:call(myapp_worker, request, [bad]) which triggers the erlang:error/1 in process/1. The supervisor logs a restart and the worker’s state is re‑initialized via init/1.
6. Verify the restart count increases in the observer or via supervisor:count_children(myapp_sup).

Limitations and Practical Checks

• Over‑supervising can mask bugs; ensure the worker’s state can be rebuilt from init/1 or is idempotent before relying on automatic restarts.
• Using a restart strategy other than one_for_one (e.g., one_for_all) when isolation is required can cause unrelated workers to terminate together.
• Synchronous calls (gen_server:call/2) inside the worker that block waiting for a resource held by the supervisor can lead to deadlocks during a restart attempt. Prefer asynchronous casts or time‑outs.
• Practical way to confirm the design behaves as intended: after a forced crash, observe that the worker’s PID changes (new process) while the supervisor PID stays the same, and that any in‑flight calls return {error, timeout} or are retried by the caller.