Why RabbitMQ Messages Disappear After a Restart

The on-call page arrives at 03:00: after a planned RabbitMQ restart, a task queue is empty and dozens of image-processing jobs are missing. The queue was declared in code, the producer logged success, and the consumer is idle. What disappeared was the assumption that a declared queue automatically survives a broker restart.

In RabbitMQ, queue durability and message persistence are two separate settings. A durable queue definition survives a restart, but only messages marked persistent are written to disk before the broker confirms receipt to the producer. If either side is missing, a restart can leave the queue empty or the messages gone.

This post explains how to configure both sides, verify persistence after a restart, and understand the throughput trade-off.

Durable queue versus persistent message

A durable queue is a queue definition stored to disk, so the queue itself reappears after the broker restarts. A persistent message is a message with delivery_mode set to 2; RabbitMQ writes it to disk before acknowledging the publish to the producer. Non-persistent messages use delivery_mode 1 and live only in memory.

Durability works only when all three are true:

• The queue was declared with durable=true.
• The message was published with delivery_mode=2.
• The consumer acknowledges messages manually and only after processing finishes.

If the queue is durable but the message is transient, the queue survives the restart but the message does not. If the message is persistent but the queue is transient, the queue definition is lost on restart and the message is lost with it.

Worked example: publish and consume a persistent message

The following Python example uses the pika client. Replace the placeholders with your broker host, queue name, and credentials.

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters(host="RABBITMQ_HOST")
)
channel = connection.channel()

# Declare a durable queue. The queue definition survives a broker restart.
channel.queue_declare(queue="QUEUE_NAME", durable=True)

# Publish a persistent message. delivery_mode=2 tells RabbitMQ to write it to disk.
channel.basic_publish(
    exchange="",
    routing_key="QUEUE_NAME",
    body=b"image-processing-job-42",
    properties=pika.BasicProperties(delivery_mode=2)
)

# Consume with manual acknowledgment.
def on_message(ch, method, properties, body):
    print(f"Received {body}")
    # Process the job here.
    ch.basic_ack(delivery_tag=method.delivery_tag)

channel.basic_consume(queue="QUEUE_NAME", on_message_callback=on_message)
channel.start_consuming()

Run this script on a machine with network access to the RabbitMQ node. The user running the script needs permission to publish to and consume from the virtual host. If queue_declare is called by both producer and consumer, both should pass durable=True; declaring an existing queue with mismatched durability raises a channel error.

Verify persistence after a restart

To confirm the configuration works, publish a persistent message, stop the RabbitMQ node, start it again, and check whether the message remains.

1. Publish one persistent message to QUEUE_NAME using the example above.
2. On the broker node, stop the service. For example, on a systemd host:

sudo systemctl stop rabbitmq-server

This command requires root or sudo access on the broker host. Stopping the broker interrupts connected publishers and consumers; plan for a brief outage.

1. Start the broker again:

sudo systemctl start rabbitmq-server

1. Check the queue flag and message count:

sudo rabbitmqctl list_queues name durable messages

Expected result: a row for QUEUE_NAME with true in the durable column and a message count of at least one until a consumer acknowledges it. If the count is zero and no consumer ran, persistence was not applied.

Trade-offs and limitations

Persistence trades throughput for safety. Writing every message to disk adds I/O latency and can reduce publish rates compared with transient messages. Under heavy load, a slow disk can also trigger RabbitMQ's disk free-space alarm, which blocks publishers until space is freed.

Persistence protects against a broker process restart, but not against disk failure or loss of the node itself. For that, you need mirrored classic queues or quorum queues with replication across nodes. Persistence also does not prevent redelivery after a consumer crash; that depends on manual acknowledgments and requeue behavior.

Checklist before you deploy

• Declare queues with durable=True in both producer and consumer code.
• Set delivery_mode=2 on every message that must survive a restart.
• Use manual acknowledgments and acknowledge only after work completes.
• Monitor disk usage and the RabbitMQ disk alarm status.
• Verify with a restart test in staging before changing production queues.

Durable queues and persistent messages are not the same thing, and RabbitMQ will not warn you when only one is configured. Check both settings, restart the broker in a test environment, and confirm the message is still there before you rely on the queue for critical work.