Idempotent Producer deduplication state after producer restart

Apache Kafka's idempotent producer uses a Producer ID (PID) and sequence numbers to prevent duplicate messages during retry cycles. This mechanism ensures that if a broker fails to acknowledge a write but the message was actually persisted, subsequent retries are discarded by the broker.

The deduplication state is tied to the lifecycle of the producer session. When a producer instance is restarted or a new producer is initialized, a new PID is assigned by the broker. This creates a gap in the continuity of the sequence tracking for a specific partition.

Given that idempotence is scoped to a single session, there is uncertainty regarding the behavior of writes that were in-flight or pending during a crash. If a producer restarts and resends the same logical data, the broker treats it as a new PID.

• Does the broker maintain any mechanism to link a new PID to a previous session's sequence numbers for the same client?
• What is the expected behavior for deduplication when a producer restart occurs between a failed acknowledgment and a retry?