Cairo query cancellation: does a hard timeout during shuffle guarantee intermediate state cleanup?

Direct answer

No. A hard timeout during a shuffle stage in Cairo (QuestDB’s distributed SQL engine) is not documented to guarantee immediate release of intermediate shuffle state on worker nodes. Cancellation is cooperative: the coordinator signals workers, but a long-running transformation typically continues until the next checkpoint or batch boundary before its execution context is torn down and intermediate buffers are released.

Confirmed behavior versus likely mechanism

Confirmed: Cairo does not publish a hard guarantee that a mid-shuffle timeout frees intermediate state at the instant the timeout fires. Client-side or gateway timeouts may abandon the query from the caller’s perspective while server-side execution continues until the engine notices the cancellation signal.

Likely, not confirmed: intermediate shuffle memory is owned by the per-query execution context on each worker and is released when that context closes. Under a timeout, that close may be deferred to the next checkpoint or query end rather than occurring at the timeout instant. Spilled shuffle artifacts, if any, may also persist until cleanup runs.

Configuration

Timeout handling has moved from session/driver-level controls toward query-level and server-side parameters across releases. Specific parameter names are version-sensitive and cannot be confirmed from memory, so do not assume keys without checking the documentation for your deployed version. No documented execution-engine parameter is known to force immediate mid-stage cleanup of shuffle state.

Verification steps

1. Check the QuestDB documentation for your deployed version under query cancellation, query timeout, and server configuration; distinguish session-level PG-wire timeouts from server-side query-level timeouts.
2. Run a controlled test with a long shuffle-heavy query, such as a large join or group-by, trigger cancellation mid-shuffle using pg_cancel_backend() or a driver timeout, and observe worker RSS and any spill or temp files immediately after versus after the next checkpoint.
3. Inspect server logs at cancellation time for messages about interrupted execution, context close, or buffer release.
4. If a gateway or proxy enforces the timeout, verify whether it sends a real cancel request to the server or merely closes the client connection; only the former should trigger engine-side cleanup.

Missing diagnostic detail

Which QuestDB or Cairo version is deployed? Exact cancellation semantics, configuration keys, and cleanup timing vary by version, and this detail changes which documentation and parameters apply.

Treat this as a conservative draft for human review until the deployed version’s documentation and a controlled cancellation test confirm the behavior.