Helm and Kubernetes: Atomic Upgrade Rollback and Hook Resource Persistence

Atomic Rollbacks and Hook Lifecycle

No, the --atomic flag does not trigger the deletion of hook resources. While --atomic ensures the release version rolls back to the previous successful revision upon failure, it does not manage the lifecycle of resources created by Helm hooks. These resources are managed independently of the release manifest.

Why Hook Resources Persist

In Helm 3.x, resources created via hooks (such as Jobs or Pods) are not tracked as part of the release's state. When a hook fails or times out, triggering an atomic rollback, Helm preserves the failed hook resource in the cluster. This behavior is intentional, allowing operators to retrieve logs (kubectl logs) and describe the resource (kubectl describe) to diagnose why the upgrade failed.

Managing Hook Cleanup

Helm does not distinguish between a "failed" hook and a "successful" one for the purpose of automatic purging during a rollback. Instead, cleanup is governed exclusively by the helm.sh/hook-delete-policy annotation. To automate the removal of hook resources regardless of the --atomic status, you must define a deletion policy in the resource metadata.

Common policy options include:

• before-hook-creation: Deletes the previous hook resource before starting a new one.
• hook-succeeded: Deletes the resource only if the hook completes successfully.
• hook-failed: Deletes the resource only if the hook fails.

Verification Steps

To verify this behavior in your environment, you can deploy a Job hook designed to timeout:

1. Create a chart with a Job annotated as a post-install or pre-upgrade hook that sleeps longer than your timeout value.
2. Execute the upgrade:

   helm upgrade --install my-release ./my-chart --atomic --timeout 2m0s

3. Verify the release version has rolled back:

   helm list

4. Confirm the hook resource still exists:

   kubectl get jobs

Note: Please specify if you are using a custom hook-delete-policy, as this would override the default persistence behavior described above.