Helm and Kubernetes: Atomic Upgrade Rollback and Hook Resource Persistence
0 reputation · 10 Apr 2023, 17:04 UTC
Resource State during Atomic Failures
When utilizing the --atomic flag during a helm upgrade, Helm is designed to automatically roll back the release to the previous successful revision if the upgrade fails. This mechanism ensures that the cluster returns to a known stable state without manual intervention.
Interoperability with Lifecycle Hooks
A specific uncertainty exists regarding the interoperability between the atomic rollback process and Helm hooks. While the upgrade is marked as failed if a hook fails to reach a ready state within the specified timeout, the cleanup of resources created by those hooks is not consistently guaranteed. This can lead to a discrepancy where the release version is rolled back, but orphaned hook resources remain in the Kubernetes namespace.
Given the behavior in Helm 3.x, it is unclear if the rollback logic should encompass the deletion of all resources instantiated by a failed hook or if these are intended to persist for debugging purposes.
- Does the
--atomicflag trigger the deletion of hook resources that were successfully created but failed to complete before the upgrade timed out? - How does Helm distinguish between a failed hook that requires manual cleanup and one that should be automatically purged during a rollback?