Using Redis Pub/Sub for Fire-and-Forget Messaging: Setup, Limits, and Checks

Quick answer: when to choose Redis Pub/Sub

If you need to broadcast short-lived events to many listeners without storing them, Redis Pub/Sub provides a fire‑and‑forget channel. Publishers send a message to a channel; any client currently subscribed receives it instantly, and the message is discarded as soon as it has been pushed to all connected subscribers. This makes Pub/Sub ideal for real‑time notifications, chat rooms, or lightweight service‑to‑service signaling where persistence is not required.

Worked configuration: two‑terminal demo

Open two terminal windows (or two tabs) and connect each to the same Redis instance using redis-cli. No special configuration is needed; the default 6379 port works.

1. In the first terminal, subscribe to a channel:

SUBSCRIBE notifications

The connection switches to subscriber mode and will wait for messages.

2. In the second terminal, publish a message:

PUBLISH notifications "User 42 logged in"

You should see the message appear immediately in the first terminal:

1) "message"
2) "notifications"
3) "User 42 logged in"

3. To illustrate the fire‑and‑forget nature, open a third terminal, subscribe after the publish, and you will receive nothing:

SUBSCRIBE notifications
# (no output appears because the earlier message was discarded)

These commands show the core lifecycle: SUBSCRIBE puts a client into a special state where only Pub/Sub messages are accepted; PUBLISH pushes the payload to all currently matched subscribers; the server does not retain the message after the push.

How the mechanism works internally

When a client runs SUBSCRIBE, Redis marks the connection as a subscriber and adds it to an internal list for each requested channel (or pattern). On PUBLISH, the server looks up that list, serializes the message into the output buffer of each matching client, and then discards the payload. No RDB or AOF entry is created, and no acknowledgment is expected from the subscriber.

Pattern subscriptions (PSUBSCRIBE) work similarly: the server checks each pattern against the channel name using a simple glob match before delivering the message.

Limits and common mistakes

1. No persistence – message loss

If no subscriber is online when a message is published, the message is gone forever. This is often surprising when developers expect a "best‑effort" queue. To detect missed messages, you can monitor the pubsub_channels metric via INFO or use a separate persistent store (e.g., a Redis Stream) for critical events.

2. Slow consumers cause buffer bloat

Each subscriber’s output buffer grows if the client does not read messages fast enough. When the buffer exceeds client-output-buffer-limit pubsub (default 32 MB), Redis forces the connection closed with ERR Client output buffer limit exceeded. To avoid this:

• Ensure subscribers process messages promptly (e.g., use async I/O or a worker pool).
• Tune the limit in redis.conf if your workload can tolerate brief bursts: client-output-buffer-limit pubsub 64mb 32mb 60.
• Check buffer usage with CLIENT LIST and look at the obl field.

3. Cluster broadcast overhead

In Redis Cluster, a PUBLISH is forwarded to every node, which then forwards to its local subscribers. As the node count rises, network traffic grows roughly O(N) for each publish, potentially saturating inter‑node links. Mitigation strategies:

• Channel‑sharding: split high‑volume topics across multiple logical channels and let each subscriber listen only to its shard.
• Consider Redis Streams or an external broker (e.g., Apache Kafka) when you need guaranteed delivery or high fan‑out at scale.

4. Misusing MONITOR for debugging

Running MONITOR on a production instance adds significant overhead because every command is duplicated to the monitor client. Use it only on a test replica or with a short‑lived session.

Practical verification steps

To confirm that your Pub/Sub setup behaves as fire‑and‑forget:

1. Start a subscriber: SUBSCRIBE test
2. In another session, publish while no one is subscribed: PUBLISH test "lost"
3. Now subscribe again: SUBSCRIBE test – you should see no message.
4. Check buffer health: CLIENT LIST and verify that the obl (output buffer length) stays well below the limit for each Pub/Sub client.

If you observe growing obl values or frequent disconnections, address the slow‑consumer issue before scaling further.

Takeaway

Redis Pub/Sub gives you a simple, low‑latency fire‑and‑forget broadcast channel when you can tolerate lost messages and need to avoid the complexity of persistent queues. By monitoring subscriber health, respecting output‑buffer limits, and understanding cluster‑wide broadcast costs, you can safely use it for real‑time notifications, presence updates, or lightweight event propagation.