Publisher Confirms and Consumer Prefetch Interoperability Under Concurrent RabbitMQ Load

Direct Answer

The interaction between publisher confirms and consumer prefetch under concurrent load does compound end-to-end latency, but the dominant factor depends on queue type and configuration. Under RabbitMQ 3.12+, the key findings are:

1. Publisher confirm latency scales linearly with concurrent publisher count when confirms are not batched, due to sequential acknowledgment round-trips. Each publisher waits for an individual ack, creating a bottleneck at high concurrency.
2. High prefetch values amplify quorum queue commit latency because messages accumulate faster than Raft consensus can commit them, increasing tail latency and memory pressure.
3. Batching confirms (100-1000 messages per batch) reduces ack overhead by approximately 80% under concurrent load, but this trades per-message durability for throughput.

Queue-Type Trade-offs

Quorum queues add ~1-3ms per-message commit latency due to Raft consensus. With high prefetch, this compounds as uncommitted messages queue in memory. Classic queues avoid replication latency but offer no fault tolerance. For latency-sensitive workloads, classic queues with batched confirms may outperform quorum queues, but at the cost of durability.

Recommended Actions

1. Enable batched confirms using confirm-select with batch sizes of 100-1000. This is the most impactful optimization for concurrent publishers.

2. Set consumer prefetch to 10-50 per consumer. Higher values increase broker memory usage and latency variance without proportional throughput gains. 3. Monitor queue depth with rabbitmqctl list_queues messages_ready messages_unacknowledged. If messages_ready grows unbounded, consider flow-control tuning or consumer scaling. 4. For latency-critical paths, evaluate classic queues with batched confirms. For durability-critical paths, use quorum queues with lower prefetch and accept higher commit latency.Verification Steps

Run a controlled load test with 10-100 concurrent publishers sending 10K messages each. Measure p99 latency with confirms enabled/disabled and varying batch sizes. Compare latency percentiles between quorum and classic queues under identical configurations. Enable RabbitMQ firehose or message tracing to capture end-to-end publish-to-consume latency distribution.

Unresolved Consideration

What is your message durability requirement? If losing messages during broker failure is unacceptable, batching confirms introduces risk that must be mitigated with upstream retry logic or dead-letter queues. If durability is not required, disabling confirms or using classic queues with batching provides the lowest latency path.