Kafka consumer group rebalancing behavior when max.poll.interval.ms times out during cancellation

Answer

If max.poll.interval.ms elapses while a shutdown/wakeup is in progress, the Kafka consumer is marked dead by the coordinator and a group rebalance is triggered before the application can finish processing the WakeupException. The exception may still be thrown on the next poll(), but the consumer has already left the group.

Likely explanation

The coordinator tracks the time between successive poll() calls. When the interval exceeds max.poll.interval.ms, it assumes the instance has crashed and sends a LeaveGroup request, initiating a rebalance. This check runs independently of the client thread, so a concurrent consumer.wakeup() cannot prevent the timeout once the timer has fired.

Confirmed facts (based on Kafka client behavior 2.0+)

• A WakeupException is thrown only when poll() is called after consumer.wakeup() has been invoked.
• The coordinator’s session timeout (session.timeout.ms) and heartbeat mechanism are separate; max.poll.interval.ms guards against processing stalls, not network failures.
• If wakeup() is called before the interval expires, the next poll() returns the exception, allowing the application to break the loop and close the consumer cleanly, thus avoiding a rebalance.
• If wakeup() occurs after the interval has already elapsed, the consumer has already been removed; the exception is still delivered but the rebalance has proceeded.

Steps to ensure predictable shutdown in this scenario

1. Estimate the worst‑case processing time plus any shutdown cleanup.
2. Set max.poll.interval.ms to a value safely above that estimate (e.g., 2× the expected maximum).
3. Before beginning shutdown, invoke consumer.wakeup() from another thread.
4. In the poll loop, catch WakeupException, break the loop, and then call consumer.close().
5. Verify that the loop exits before the interval timer would fire; if not, increase the interval or reduce processing time.

If you observe rebalances despite these steps, check the consumer’s heartbeat latency or any prolonged blocking calls that may be preventing the poll loop from advancing.