Publisher Confirms and Consumer Prefetch Interoperability Under Concurrent RabbitMQ Load
0 reputation · 13 Nov 2024, 23:29 UTC
The goal is to determine whether publisher confirms and consumer prefetch combine to increase end-to-end latency when multiple producers and consumers operate concurrently against the same RabbitMQ queue.
Publisher confirms require the broker to acknowledge each published message, which can block senders at high concurrency unless confirms are batched. Consumer prefetch limits how many unacknowledged messages a consumer can hold, so a high prefetch may queue messages faster than consumers process them. Quorum queues add Raft commit latency, while classic queues do not replicate by default, making the interaction queue-type dependent.
The unresolved decision is whether to lower prefetch, batch confirms, or switch queue types when concurrent traffic shows latency spikes, and how much each factor contributes under RabbitMQ 3.12 or later.
- Does publisher-confirm latency grow linearly with concurrent publisher count when prefetch is unchanged?
- Does a high prefetch multiplier compound quorum-queue commit latency under concurrent consume workloads?
- Should confirms be batched or disabled for latency-sensitive concurrent publishers, and does the trade-off differ between classic and quorum queues?