Does limiting old‑generation heap size with +hms prevent memory leaks in long‑running Elixir processes?

Question

The goal is to determine whether imposing a hard limit on the old‑generation heap size via the BEAM flag +hms reliably prevents the unbounded memory growth that characterizes leaks in long‑running Elixir processes. Such leaks often stem from large binaries, ETS tables, or global process dictionaries that survive minor collections and accumulate in the old generation.

Constraints include the necessity to restart the node to apply +hms, the global effect on all processes, and the risk that frequent old‑gen collections may degrade performance or merely hide the leak rather than eliminate it. It is also unclear how the flag interacts with leaks that involve resources not reclaimed by garbage collection, such as open file descriptors or native resources.

Does the +hms flag actually stop old‑generation heap growth when the leak resides in binary data that is not subject to GC? If the flag triggers premature old‑gen collections, does it merely mask the leak or reduce observable memory growth without fixing the root cause?

Tasadduq Burney · Answer

No, the +hms flag does not prevent memory leaks. It imposes a hard limit on the old-generation heap size, but it cannot stop the underlying cause of a leak or reclaim memory that is still reachable by the BEAM virtual machine.

How +hms Interacts with Memory Growth
The +hms flag defines a ceiling for the old-generation heap. When the heap reaches this limit, the VM is forced to trigger a major (full-sweep) garbage collection to attempt to reclaim space. This creates two distinct scenarios depending on the nature of the leak:

Reclaimable Leaks: If the memory growth is caused by data that eventually becomes unreachable, +hms may keep memory usage stable by forcing more frequent collections. However, this is a mitigation of symptoms, not a fix for the root cause.
  Unreclaimable Leaks: If the leak consists of data that remains reachable (e.g., an ever-growing ETS table, a process dictionary that is never cleared, or binaries referenced by long-lived processes), the major GC will fail to reclaim the memory. The VM will continue to trigger expensive full-sweeps as it hits the limit, eventually leading to an Out-of-Memory (OOM) crash.

Scope and Limitations
It is important to distinguish between the BEAM heap and other memory areas. The +hms flag does not affect the following:

Native Resources: Memory allocated via NIFs (Native Implemented Functions) or open file descriptors.
  Binary Heap: Large binaries (typically > 64 bytes) are stored in a separate global binary heap, not the process-specific old-generation heap.
  ETS Tables: While references to ETS tables are on the heap, the data within the tables resides in a separate memory area.

Verification Steps
To verify if +hms is masking a leak or failing to stop one, follow these steps:

Baseline: Run the node without +hms and monitor the old-generation heap size using :erlang.memory() or the observer tool. Confirm unbounded growth.
  Apply Limit: Restart the node with +hms set to a value slightly above your expected steady-state usage.
  Analyze Behavior:
    
      If memory oscillates around the limit and CPU usage spikes (due to frequent major GCs), the leak is still present but being throttled.
      If the node crashes with an OOM error despite the limit, the leak involves unreachable-resistant data or non-heap resources.

Missing Diagnostic: To provide a more specific recommendation, please clarify if your memory growth is appearing as binary, ets, or process memory in your :erlang.memory() output.

Does limiting old‑generation heap size with +hms prevent memory leaks in long‑running Elixir processes?

1 answer

How +hms Interacts with Memory Growth

Scope and Limitations

Verification Steps

Your perspective belongs here.

0 question comments