Provider‑declared retryable actions versus Terraform retry block: avoiding duplicate writes on transient errors

Question

Goal
Ensure that Terraform retries a provider operation after a transient error without creating duplicate resources when the underlying API call is not idempotent.
Constraints and uncertainty

Providers can mark individual CRUD actions as retryable via the SDK’s Retryable field, which assumes the provider implements idempotency for those actions.
If an action is not marked retryable, Terraform performs no automatic retries; users must handle retries externally, which can cause duplicate writes if the action is not idempotent.
The experimental retry block introduced in Terraform 1.5 lets users configure attempt counts and back‑off, but it does not provide idempotency guarantees—safety still depends on the provider.
An open discussion (issue #31284) considers whether Terraform should automatically generate idempotency tokens for AWS‑style APIs on retryable actions, shifting the burden from providers to core, but the proposal is still unresolved.

Should Terraform automatically inject idempotency tokens for retryable actions when a provider does not supply them?
Is it safer to rely on provider‑declared retryable flags or to use the experimental retry block with external idempotency handling?
What verification approach can confirm that a retry does not produce duplicate writes given the current Terraform behavior?

Tasadduq Burney · Answer

Answer
1. Do not rely on Terraform to inject idempotency tokens automatically. Terraform’s core retry logic only re‑invokes the provider’s CRUD function; it does not add any idempotency guarantees. If a provider does not expose an idempotent create operation, retries will create duplicates.
2. Prefer provider‑declared retryable actions when the provider implements idempotency. The Retryable field in the SDK is meant to signal that the provider’s implementation is safe to retry. The experimental retry block is useful for tuning back‑off or limiting attempts, but it does not add idempotency itself.
3. Verification strategy:

Run terraform plan after a transient failure and inspect the state file for duplicate resource IDs.
Enable TF_LOG=DEBUG to confirm that Terraform re‑calls the same provider action and that the provider’s create function is executed again.
If the provider is open source, review the create implementation to ensure it performs an existence check or uses an idempotency token.
Apply a retry block with reasonable max_attempts and delay values; observe that Terraform stops retrying after the first successful response or after a non‑retryable error.

These steps confirm that a retry does not result in a second, distinct resource. If a duplicate appears, the provider’s create logic must be adjusted to be truly idempotent.
Further Clarification Needed
To tailor the recommendation more precisely, please provide:

The specific provider and resource type you are using.
The transient error code (e.g., HTTP 502, timeout) that triggered the retry.

Provider‑declared retryable actions versus Terraform retry block: avoiding duplicate writes on transient errors

Goal

Constraints and uncertainty

1 answer

0 question comments

Answer

Further Clarification Needed

Your perspective belongs here.