Managing Lifecycle Tasks with Helm Hooks

Solving the Dependency Gap in Kubernetes Deployments

Standard Kubernetes manifests are declarative, meaning they describe a desired state but do not guarantee the order of operations. This creates a problem when a deployment requires a specific task—such as a database schema migration or a secret generation—to complete successfully before the application pods start. If the application starts before the database is ready, the pods may enter a CrashLoopBackOff state.

The solution is Helm Hooks. Hooks allow you to trigger specific Kubernetes resources (usually Jobs) at precise points in the release lifecycle. By using hooks, you can ensure that prerequisite tasks are executed and verified before Helm proceeds with the rest of the installation or upgrade.

How Helm Hooks Work

Helm identifies a resource as a hook by looking for the helm.sh/hook annotation in the manifest. When Helm processes a chart, it separates resources marked as hooks from the rest of the release. It executes these hooks in a specific sequence based on the lifecycle event (e.g., pre-install) and the assigned weight.

Practical Example: Database Migration Job

Below is a configuration for a migration job that must run before any application pods are deployed. This file would typically reside in templates/hooks/migration-job.yaml.

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ .Release.Name }}-db-migrate
  annotations:
    # Run this before installation and before upgrades
    "helm.sh/hook": pre-install,pre-upgrade
    # Execute this hook before others with higher weights
    "helm.sh/hook-weight": "-5"
    # Delete the job only if it completes successfully
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  template:
    spec:
      containers:
        - name: migrate
          image: my-migration-tool:1.2.0
          command: ["/bin/sh", "-c", "/app/migrate.sh"]
      restartPolicy: OnFailure

Key Configuration Parameters

• helm.sh/hook: Defines the trigger. Common values include pre-install, post-install, pre-upgrade, post-upgrade, and pre-delete.
• helm.sh/hook-weight: An integer that determines execution order. Hooks with lower weights (e.g., -5) run before hooks with higher weights (e.g., 10).
• helm.sh/hook-delete-policy: Determines when the hook resource is removed from the cluster. Options include before-hook-creation (cleans up previous runs), hook-succeeded, or hook-failed.

Execution Logic and Limitations

It is critical to understand that hooks are not part of the release's managed state. If you run helm list, the hook resources are not tracked as part of the release manifest. This means that if a hook is not deleted via a delete policy, it will remain in the cluster even after the release is deleted, unless you use a pre-delete hook to clean it up.

Critical Constraints

• Blocking Nature: A pre-install or pre-upgrade hook must exit with a status code of 0. If the Job fails, Helm marks the entire release as failed and will not deploy the remaining resources.
• RBAC Requirements: Hooks run using the service account provided in the Job spec. If your migration job needs to create secrets or modify ConfigMaps, you must include the necessary Role and RoleBinding in your chart.
• Idempotency: Because hooks can run during both installation and upgrades, the logic inside the container must be idempotent. Running a migration script twice should not result in an error or duplicate data.

Common Pitfalls

Mistake	Consequence	Fix
Omitting hook-weight	Unpredictable execution order when multiple hooks exist.	Assign explicit weights (e.g., -10, 0, 10).
Assuming resource readiness	pre-install hooks run before any other resource is created.	Use post-install if the hook needs to interact with a deployed service.
Missing delete-policy	Job pods remain in the cluster, potentially blocking future runs.	Use hook-succeeded or before-hook-creation.

Verification and Diagnostics

To verify hook configuration without deploying to a cluster, run the following command from your local terminal:

helm template my-release ./my-chart | grep "helm.sh/hook"

This confirms that the annotations are correctly rendered in the YAML output.

To diagnose a failing hook during a live deployment, use kubectl to inspect the Job logs. Since hooks are created in the release namespace, run:

# Find the job name
kubectl get jobs -n <namespace>

# Check the logs of the pod created by the job
kubectl logs job/<job-name> -n <namespace>

If the Job status shows Succeeded: 1, Helm will proceed. If it shows Failed, the release will stop, and you must fix the underlying issue and trigger the upgrade/install again.

Rollback Considerations

Because hooks change the state of external systems (like databases), helm rollback does not automatically undo the actions of a hook. If a pre-upgrade migration fails and triggers a rollback, the database may remain in a partially migrated state. You must implement a manual recovery plan or a specific pre-rollback hook to handle data reversion.