Erlang gen_server: When the Synchronous Loop Becomes Your Bottleneck

You have an Erlang gen_server handling user requests. Traffic rises, latency climbs, and the CPU sits idle. The problem is rarely the algorithm; it is a synchronous handle_call/3 callback holding the process hostage while it waits for a slow operation. The takeaway: gen_server's single-threaded event loop is a consistency feature, not a throughput feature. Your engineering decision is knowing when to keep requests serialized and when to move them off the critical path.

Why gen_server Serializes Requests

OTP's gen_server behavior gives each server process a mailbox. Messages arrive in order, and the process handles them one at a time. When a client calls gen_server:call/2, the server executes handle_call/3, produces a reply, and only then reads the next message. This removes the need for manual locks around shared state because there is exactly one thread of execution per server.

That serialization is also the limitation. If handle_call/3 runs a slow database query, calls an external HTTP service, or loops over a large dataset, every other request waits in the mailbox. The process is not crashed; it is busy. Under load, the mailbox grows, callers time out, and the supervisor sees a healthy process that is effectively unavailable.

Worked Example: Moving Slow Work Off the Loop

Consider a cache server that fetches a value on a miss. The naive version blocks the entire loop:

-module(cache_server).
-behaviour(gen_server).

%% API
start_link() -> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
lookup(Key) -> gen_server:call(?MODULE, {lookup, Key}).

%% Callbacks
init([]) -> {ok, #{}}.

handle_call({lookup, Key}, _From, State) ->
    case maps:find(Key, State) of
        {ok, Value} ->
            {reply, Value, State};
        error ->
            %% BLOCKING: external fetch stalls the whole server
            Value = slow_external_fetch(Key),
            {reply, Value, maps:put(Key, Value, State)}
    end.

Every miss stops all traffic. Three refactoring patterns preserve the gen_server contract while removing the stall.

Pattern	Mechanism	Best for	Trade-off
Async cast + poll	handle_cast starts work; client polls or is notified	Fire-and-forget updates	Client complexity; no direct reply guarantee
Worker process	handle_call spawns/monitors a worker; server stays responsive	Slow per-request operations	Process overhead; need to handle worker crashes
Sharded gen_servers	Multiple server processes behind a dispatcher	High-throughput key spaces	State partitioning logic

For the cache example, a worker-process refactor keeps the synchronous API but moves the blocking fetch off the server loop:

handle_call({lookup, Key}, From, State) ->
    case maps:find(Key, State) of
        {ok, Value} ->
            {reply, Value, State};
        error ->
            %% Spawn a worker; server returns to its mailbox immediately
            {_Pid, Ref} = spawn_monitor(fun() ->
                Value = slow_external_fetch(Key),
                exit({fetch_result, Value})
            end),
            {noreply, State#{pending_from => From, worker_ref => Ref}}
    end;

handle_info({'DOWN', Ref, process, _Pid, {fetch_result, Value}}, State)
  when Ref =:= maps:get(worker_ref, State) ->
    From = maps:get(pending_from, State),
    gen_server:reply(From, Value),
    {noreply, maps:without([pending_from, worker_ref], State)};
handle_info({'DOWN', _Ref, process, _Pid, Reason}, State) ->
    %% Worker crashed; supervisor policy decides restart
    {stop, {worker_crash, Reason}, State}.

Here the server only coordinates. The expensive work happens in a separate process that the supervision tree can restart independently if it crashes.

Hidden State and Hot Code Upgrades

Research cautions against misusing global state inside callbacks. When a gen_server reaches into hidden global dependencies such as the process dictionary or application environment, it breaks the isolation that makes hot code upgrades predictable. The code:replace/4 mechanism, used by release_handler during release upgrades, expects state to travel with the process. Hidden global dependencies mean an upgraded callback may read data the old version never touched, or write data the new version cannot find. Keep state in the gen_server's own state tuple and pass it explicitly.

Verifying the Bottleneck Before You Refactor

Before changing code, confirm the bottleneck. Start the observer from an Erlang shell on the node by running observer:start(). This requires the runtime_tools and GUI libraries to be available, and the shell must have permission to connect to the node. Inspect the target process with process_info(whereis(cache_server)) and check message_queue_len and current_function. A growing queue with current_function stuck in slow_external_fetch confirms the hypothesis. For deeper tracing, redirect trace output to a file using dbg:tracer(process, {handler, fun(_,_) -> ok end}) or similar, but be aware that tracing adds overhead and can alter timing.

Async and multi-process patterns add complexity. You now manage worker lifecycles, reply correlation, and partial failure. Sharded servers require consistent hashing or a dispatcher. For low-traffic systems where requests are inherently fast, the naive synchronous gen_server remains the simplest correct choice. Measure first.

Treat gen_server as a state machine with a single turnstile, not a general-purpose worker pool. When handle_call/3 does more than mutate memory and return, that is your signal to refactor. Start with process_info/1 to verify the queue is the problem, then choose the lightest pattern that unblocks the loop: a cast for fire-and-forget, a monitored worker for slow replies, or sharding for volume. Keep state local, and your supervision tree will keep the system recoverable.