Idempotency-Key header vs. etag/resourceVersion deduplication for Rancher API retries

The Strongest Guarantee for Production

For the strongest guarantee against duplicate writes across various failure scenarios, enabling the Idempotency-Key feature flag (Rancher v2.6+) is the superior approach. While resourceVersion is critical for preventing concurrent update conflicts, it cannot prevent the creation of duplicate resources during a retry of a POST request.

Comparison of Deduplication Mechanisms

The choice depends on whether you are creating a new resource or updating an existing one:

Mechanism	Primary Use Case	Prevents Duplicate POST?	Prevents Lost Updates?	Dependency
Idempotency-Key	Resource Creation	Yes	No	Rancher v2.6+ & Feature Flag
resourceVersion	Resource Updates	No	Yes	Kubernetes API Server
Client-side Log	General Retries	Partial	No	Persistent Local Storage

Analysis of Specific Constraints

• Network Partitions: The Idempotency-Key prevents duplicate writes during network partitions. If a client sends a POST request, the server processes it but the response is lost due to a partition, the client's subsequent retry with the same key will return the original success response rather than creating a second resource.
• resourceVersion Behavior: resourceVersion (and ETags) implements optimistic concurrency control. If the Kubernetes API does not increment the version (which is rare but can occur in specific custom controller implementations), an ETag-based update may be accepted even if the state has changed, or rejected if the client's version is perceived as stale. Crucially, it cannot stop duplicate POST requests because the version is only assigned after the resource is created.
• Operational Overhead: Persisting a client-side retry log requires a reliable local database or filesystem write before every API call to ensure the log survives a process crash. This introduces significant I/O overhead and complexity compared to the server-side Idempotency-Key, which shifts the state management to the Rancher API.

Verification Steps

To verify your implementation, use the following scoped tests:

1. Verify Feature Flag: Confirm the Idempotency-Key flag is active in Rancher global settings.
2. Test POST Idempotency: Send a POST request with a unique Idempotency-Key, simulate a timeout (e.g., via a proxy), and retry the exact same request. Confirm only one resource exists.
3. Test Update Conflict: Perform a PUT request using an outdated resourceVersion. The API should return a 409 Conflict.

Missing Diagnostic: Are you targeting specific Custom Resource Definitions (CRDs) managed by external controllers? If so, the behavior of resourceVersion may vary based on how those controllers handle updates.