Can Phoenix Channels avoid mailbox buildup during concurrent broadcasts?

Managing Fan-out and Mailbox Pressure

Yes, Phoenix Channels can avoid global mailbox buildup because the broadcast fan-out is not handled by a single bottleneck process. Instead, the Phoenix.PubSub system distributes messages across the cluster, and the final delivery is handled by the individual process assigned to each client connection.

How Fan-out and Ordering Work

The broadcasting process sends a message to the PubSub system, which identifies the relevant subscribers. The PubSub system then sends the message to each subscriber's individual channel process. Because the BEAM guarantees that messages sent from Process A to Process B arrive in the order they were sent, ordering is preserved per subscriber.

Latency and Scheduler Behavior

Increased subscriber counts do not typically increase per-message latency linearly for the sender, but they do introduce tail latency for the receivers. This is caused by:

• Scheduler Contention: The BEAM scheduler must wake up thousands of processes to handle the incoming message.
• TCP Backpressure: If a client's network is slow, the individual channel process's mailbox will grow because it cannot drain the messages to the TCP socket as fast as the PubSub system delivers them.

When to Partition Application State

While the broadcast mechanism is distributed, your application state might not be. You should partition state across multiple GenServers (sharding) when:

• A single GenServer is used to track global presence or state for all subscribers.
• The process mailbox of that state-manager becomes a sequential bottleneck, causing requests to queue.
• You observe high CPU usage on a single core while others remain idle during state updates.

Verification Steps

To determine if you are experiencing mailbox buildup, you can monitor the queue length of specific processes during a load test:

# Check the message queue length of a specific process PID
:process.message_queue_len(pid)

If the queue length grows indefinitely for specific client PIDs, you have a slow-client problem. If it grows for a single state-management PID, you have a bottleneck that requires sharding.

Diagnostic Detail Needed: Are you observing the buildup in the individual client processes or in a custom GenServer used to manage the broadcast logic?