RabbitMQ Reliability: Publisher Confirms and Manual Consumer Acknowledgments

The problem: delivery is not the same as responsibility

Publishing a message to RabbitMQ does not mean the message is safe. The broker may accept it and then fail before persisting it, or a consumer may receive it and crash before finishing work. Without explicit confirmation on both sides, message loss is silent.

The practical takeaway is to treat RabbitMQ as a durable handoff point, not a guarantee. The producer should wait for a publisher confirm before considering a message sent. The consumer should use manual acknowledgements and only acknowledge after its side effects are committed. A bounded prefetch keeps one consumer from hoarding unacknowledged messages.

How publisher confirms work

Publisher confirms are a channel-level extension to AMQP 0-9-1. When a channel enters confirm mode, the broker tracks every publish on that channel and replies with basic.ack when it accepts the message or basic.nack if it cannot. The confirm tells the producer the broker has taken responsibility, typically after writing to a durable queue. It does not mean any downstream service has processed the message.

How manual consumer acknowledgements work

By default, many client libraries use automatic acknowledgements: the broker removes a message from the queue as soon as it is delivered. If the consumer crashes immediately after delivery, the message is lost. With manual acknowledgements, the broker keeps the message in an unacknowledged state until the consumer sends basic.ack. If the consumer channel closes before acknowledging, RabbitMQ redelivers the message to another consumer or to the same consumer when it reconnects. This gives at-least-once delivery, so message handlers must be idempotent.

Bounded prefetch

basicQos sets the prefetch count, limiting how many unacknowledged messages the broker pushes to a consumer channel. A value of zero means unlimited. On busy consumers, an unbounded prefetch can cause head-of-line blocking, where one slow consumer holds many messages while other consumers sit idle.

Worked configuration: Java client

Assumptions: RabbitMQ 3.12 or later, a durable queue named orders, persistent messages, and the Java client 5.x. Method names vary by client language and version; check your client reference before adapting this code.

Run the producer code on the application host. The publishing application needs permission to publish to the exchange and to publish persistent messages.

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.AMQP;

// Producer: enable confirms and publish persistently
Channel channel = connection.createChannel();
channel.confirmSelect();

AMQP.BasicProperties props = new AMQP.BasicProperties.Builder()
    .deliveryMode(2) // persistent message
    .build();

byte[] body = \"{\\"orderId\\":123}\".getBytes();
channel.basicPublish(\"orders-exchange\", \"orders.key\", props, body);

// Block until the broker confirms or rejects this publish
boolean ok = channel.waitForConfirmsOrDie(5000);
if (!ok) {
    throw new IllegalStateException(\"Broker did not confirm within timeout\");
}

waitForConfirmsOrDie blocks the publishing thread and throws if the broker nacks or the timeout expires. In high-throughput producers, use asynchronous confirm listeners instead of blocking calls to avoid latency.

Run the consumer code on the worker host. The consumer needs permission to consume from the queue and to acknowledge messages.

import com.rabbitmq.client.DeliverCallback;

// Consumer: manual acknowledgement with bounded prefetch
Channel channel = connection.createChannel();
channel.basicQos(20); // at most 20 unacknowledged messages per channel

DeliverCallback deliverCallback = (consumerTag, delivery) -> {
    try {
        processOrder(delivery.getBody());
        // Commit database writes or other side effects first
        channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
    } catch (Exception e) {
        // Do not ack. Requeue or dead-letter according to your retry policy.
        channel.basicNack(delivery.getEnvelope().getDeliveryTag(), false, false);
    }
};

channel.basicConsume(\"orders\", false, deliverCallback, consumerTag -> {});

basicConsume with autoAck set to false keeps the broker from removing the message until basicAck is called. basicQos(20) bounds the number of outstanding deliveries on this channel.

Limits and common mistakes

• Publisher confirms add round-trip latency. Synchronous waits reduce throughput; asynchronous confirms are better for high-volume publishing.
• A confirm means the broker accepted the message, not that a downstream service succeeded. Do not use it as end-to-end proof of processing.
• Durable queues with persistent messages can still lose data on a hard crash if the producer never waits for the confirm.
• Leaving autoAck true causes message loss on consumer crashes, because the broker acknowledges on delivery.
• Setting prefetchCount to zero or a very high value can overwhelm a slow consumer and increase tail latency.
• Acknowledging before side effects are committed means a crash after the ack but before the commit loses work.
• Unacknowledged messages are redelivered after a channel close. Handlers must be idempotent because the same message can be processed more than once.

How to verify the behavior

Publish a test message with confirms enabled. Stop the consumer process before it calls basicAck. Restart the consumer and confirm that the same message is redelivered. This proves the broker retained the unacknowledged delivery.

On the broker host, run as a user with management or monitoring permissions:

rabbitmqctl list_queues name messages messages_unacknowledged consumers

Expected check: while the consumer is processing, messages_unacknowledged should be greater than zero. After successful processing and acknowledgement, it should drop toward zero. A persistently high value indicates consumers are not acknowledging or are crashing before ack.

Risk: rabbitmqctl reads live broker state. Avoid running it repeatedly during peak load, and restrict access to trusted operators.

Queue type considerations

Classic mirrored queues and quorum queues have different persistence and confirm semantics. Quorum queues replicate messages across nodes and are designed for stronger durability than classic mirrored queues, but confirm behavior can still vary by client and deployment. Verify which queue type your deployment uses before relying on specific durability guarantees.

Keep confirms awaited, keep acknowledgements manual, keep prefetch bounded, and keep handlers idempotent. That combination is the minimum reliable handoff pattern for RabbitMQ.