Diagnosing and Fixing Process Memory Leaks in Elixir

The Symptom: Unbounded Memory Growth

In Elixir, a memory leak rarely means a failure of the garbage collector (GC). Instead, it usually manifests as Heap Growth: a situation where a long-lived process (like a GenServer) continues to hold references to data it no longer needs, or accumulates messages faster than it can process them. This results in a gradual increase in memory usage reported by :erlang.memory() that does not plateau, even when request volume stabilizes.

Quick Diagnostic Matrix

Observation	Likely Cause	Primary Metric to Check
Steady growth in total memory; few processes.	State accumulation in GenServer.	Process Heap / Binaries
Spikes in memory during high load; slow recovery.	Mailbox overflow (Slow Consumer).	message_queue_len
High memory usage across many short-lived processes.	Timer/Process leakage.	Process count (:erlang.system_info(:process_count))

Step-by-Step Memory Investigation

To resolve a leak, you must first isolate whether the memory is held in the Process Heap (private to a process) or the Binary Heap (shared memory for binaries larger than 64 bytes). Use these steps to identify the culprit.

1. Identify the Heavy Processes

Avoid using :observer in production environments as it can introduce significant latency. Instead, use the recon library to sample processes. Run the following in the Elixir shell (IEx) with administrative permissions:

# Find the top 10 processes by memory usage
:recon.proc_count(mem=true, limit=10)

Expected Result: A list of PIDs and their memory consumption. If a single GenServer is consuming hundreds of megabytes, you have a state leak. If thousands of processes are consuming small amounts, you have a process leak.

2. Inspect the Mailbox

If a process is growing, check if it is failing to keep up with its messages. A backed-up mailbox prevents the GC from reclaiming memory associated with those messages.

# Replace PID with the identified process ID
:erlang.process_info(PID, :message_queue_len)

Risk: If the queue length is in the thousands and growing, the process is a bottleneck. Adding more memory will only delay the eventual crash (Out of Memory).

3. Analyze the State Structure

If the mailbox is empty but the heap is large, inspect the process state. Look for lists or maps that grow indefinitely (e.g., a cache that never expires or a log of events stored in the GenServer state).

Remediation Strategies

Fix A: Moving State to ETS

When a GenServer state grows large, every state update requires the BEAM to copy the updated state, increasing GC pressure. Move large datasets to ETS (Erlang Term Storage), which stores data outside the process heap.

# Example: Replacing a state map with an ETS table
# In init/1
:ets.new(:my_cache, [:set, :public, :named_table])
{:ok, %{cache_size: 0}}

# In handle_cast/2
:ets.insert(:my_cache, {key, value})
{:noreply, state}

Fix B: Implementing Selective Receive

If the mailbox is the issue, avoid receive blocks that match everything. Use selective receive to process high-priority messages first, or implement a Backpressure mechanism (such as GenStage) to limit the rate of incoming messages.

Fix C: Breaking Reference Cycles

Ensure that long-lived processes are not holding references to large binaries that are no longer needed. If you are slicing large binaries, be aware that the slice maintains a reference to the original large binary in memory. Use binary_to_list/1 or binary_copy/1 to break the link if only a small fragment is needed long-term.

Verification and Testing

To verify the fix, perform a load test using a tool like k6 or wrk while monitoring memory snapshots:

1. Capture baseline memory: :erlang.memory().
2. Apply load for 10 minutes.
3. Trigger a manual GC for the suspect process (only in staging): :erlang.garbage_collect(PID).
4. Compare current memory to baseline. If memory returns to baseline levels, the leak is resolved.

Escalation Criteria

If the following conditions persist after applying the above fixes, escalate to a BEAM performance specialist:

• Memory growth continues despite ETS migration and empty mailboxes.
• Binary heap grows while process heaps remain stable (indicates a leak in a NIF or a shared binary reference).
• The application crashes with out of memory despite low reported :erlang.memory() (indicates fragmentation or OS-level limits).