Query concurrency limit queue behavior change in Prometheus 3.0

Prometheus enforces a global query concurrency limit via --query.max-concurrency, default 20. Excess queries queue and latency appears only when concurrent requests exceed the limit. The limit applies to user queries and internal workloads such as recording rule and alerting rule evaluations.

Prometheus 3.0 introduced a new query engine with improved parallelism but retained the same concurrency flag, documented as per query-engine instance rather than per-tenant. Queue ordering is described as FIFO with no priority inversion protection, which can amplify tail latency for mixed workloads.

The design intent for timeout handling while a query is queued is unclear. It is unresolved whether a query that times out during queuing releases its reserved concurrency slot immediately or leaves the slot in an indeterminate state.

Questions:

• Does a timeout while queued free the concurrency slot immediately in Prometheus 2.x versus 3.x?
• Is queue ordering strictly FIFO across instant, range, recording and alerting evaluations?
• Are there documented guarantees about slot leakage under timeout conditions?