Choosing Shared State Storage in Erlang/OTP: ETS vs. persistent_term vs. Process Dictionary

The State Management Dilemma

In Erlang, the default pattern is to encapsulate state within a process. However, passing large state records through deep call chains or managing high-frequency lookups across thousands of processes creates bottlenecks. The challenge is selecting a storage mechanism that balances read/write latency against concurrency and memory safety.

The primary takeaway: Use ETS for mutable shared data, persistent_term for immutable global configuration, and the Process Dictionary only for temporary, process-local caching.

Storage Option Comparison

Feature	ETS Tables	persistent_term	Process Dictionary
Scope	Shared (Node-local)	Shared (Node-local)	Private (Process-local)
Mutability	Mutable	Immutable (Write-once/rarely)	Mutable
Read Speed	Fast (O(log N))	Near-Constant (O(1))	Fast (O(1))
Write Cost	Low	Very High (Global GC)	Very Low
Concurrency	High (Concurrent reads)	Highest (Lock-free reads)	None (Single process)

Trade-offs and Constraints

ETS (Erlang Term Storage)

ETS is the workhorse for shared state. It allows multiple processes to read and write to the same table. While highly flexible, it introduces ownership risks: by default, if the process that created the table crashes, the table is deleted. To prevent this, you must use ets:give_away/3 to transfer ownership to a supervisor or a long-lived process.

persistent_term

Introduced in OTP 23, persistent_term provides the fastest possible read access because the VM treats the stored term as a constant. However, updating a term triggers a global garbage collection across all schedulers. If used in a hot write path, it will cause significant system-wide latency spikes (stalls).

Process Dictionary

The process dictionary is a hidden map attached to the current process. While it avoids the overhead of passing arguments, it is generally discouraged in library code because it creates "hidden coupling," making functions harder to test in isolation and prone to collisions if multiple libraries use the same keys.

Implementation and Validation

To decide between ETS and persistent_term, you must measure the frequency of writes versus reads. The following example demonstrates how to initialize both and benchmark their read performance.

%% Run in Erlang shell (OTP 23+) 
%% Setup ETS table
Table = ets:new(my_cache, [set, public, named_table, read_concurrency]).
ets:insert(Table, {user_1, "John Doe"}).

%% Setup persistent_term
PTerm = persistent_term:put(config_key, #{version => "1.0", env => "prod"}).

%% Benchmark ETS Read
{TimeETS, _} = timer:tc(fun() -> ets:lookup(my_cache, user_1) end).

%% Benchmark persistent_term Read
{TimePT, _} = timer:tc(fun() -> persistent_term:get(config_key) end).

io:format("ETS Read: ~p us | PTerm Read: ~p us~n", [TimeETS, TimePT]).

Diagnostic Checks

• Memory Inspection: Run observer:start() to monitor the memory footprint of ETS tables. Large tables can lead to fragmentation.
• Scheduler Impact: When updating persistent_term, monitor erlang:system_info(scheduler_wall_time) to detect if the global GC is causing scheduler stalls.
• Version Check: Confirm your OTP version to ensure persistent_term and read_concurrency (OTP 26+) are available:

  erl -eval "io:format(\"OTP ~s~n\", [erlang:system_info(otp_release)]), init:stop()."

Operational Limitations

None of these mechanisms are distributed. If your application runs on a cluster, an ETS table on Node A is invisible to Node B. For cross-node state, you must implement a distribution layer using rpc:call/4 or transition to a distributed database like Mnesia. Additionally, be cautious with ETS bag or duplicate_bag types; while they allow multiple values per key, they complicate lookup logic and can increase memory consumption if not pruned regularly.