Protecting Rancher Itself: Encrypted rancher-backup Schedules to S3 and Restore Rehearsals

If the cluster running Rancher loses its Rancher management resources, the failure is not always obvious at first: downstream workloads can keep serving traffic while the inventory of clusters, users, roles and project bindings disappears. The useful takeaway is to treat Rancher as a stateful service. Use the rancher-backup operator to schedule encrypted backups of Rancher management-plane resources to S3-compatible object storage, then rehearse a restore before you need it.

What rancher-backup protects

rancher-backup installs into cattle-resources-system and adds Backup, Restore and ResourceSet resources under resources.cattle.io. A ResourceSet is the operator's declared selection of Kubernetes resources to include. The default rancher-resource-set selects Rancher-owned management state, which is the part you lose when the local cluster is damaged.

This article assumes a Rancher/rancher-backup release that exposes those CRDs and the default resource set. Field names, chart versions, UI labels and default selections are release-sensitive, so confirm them against your installed cluster before committing YAML.

Worked example: encrypted hourly backup to S3-compatible storage

Run these commands from an administration workstation using the kubeconfig for the local cluster where Rancher runs. Installing the operator and creating cluster-scoped resources normally requires cluster-admin.

1. Install the chart that matches your Rancher version:

   helm upgrade --install rancher-backup <chart-repo>/rancher-backup --namespace cattle-resources-system --create-namespace --version <rancher-backup-chart-version>

   Check that the CRDs and operator pods exist:

   kubectl get crd | grep resources.cattle.io
   kubectl -n cattle-resources-system get pods

2. Create the S3 credential secret in cattle-resources-system. Confirm the expected key names in your release; the following uses the common accessKey and secretKey names:

   apiVersion: v1
   kind: Secret
   metadata:
     name: rancher-backup-s3
     namespace: cattle-resources-system
   type: Opaque
   stringData:
     accessKey: <s3-access-key-id>
     secretKey: <s3-secret-access-key>

   Apply it with kubectl apply -f. Prefer your secret manager or a sealed-secret workflow, and do not commit real credentials.

3. Create the encryption secret referenced by the Backup. The exact key layout is release-sensitive; confirm it with kubectl explain backup.spec and your chart documentation. Store it with the same care as the backups because losing it can make otherwise intact backups unusable.
4. Create the Backup CR. Inspect the schema first:

   kubectl explain backup.spec
   kubectl explain resourceset

   Then apply an object like this, adjusting field names if your release differs:

   apiVersion: resources.cattle.io/v1
   kind: Backup
   metadata:
     name: rancher-hourly
     namespace: cattle-resources-system
   spec:
     resourceSetName: rancher-resource-set
     schedule: "0 * * * *"
     retentionCount: 24
     encryptionConfigSecretName: rancher-backup-encryption
     storageLocation:
       s3:
         credentialSecretName: rancher-backup-s3
         credentialSecretNamespace: cattle-resources-system
         bucketName: <s3-bucket-name>
         folder: <rancher-backup-folder>
         region: <s3-region>
         endpoint: <s3-endpoint-host>

   If you want a single validation run first, remove schedule and retentionCount only after kubectl explain backup.spec confirms they are optional in your release.

5. Verify the result:

   kubectl -n cattle-resources-system get backup rancher-hourly -o yaml
   kubectl -n cattle-resources-system describe backup rancher-hourly

   Look for success conditions and the recorded backup filename or location. Then check the S3-compatible bucket for an object whose size and age match the schedule. Do not assume success from the absence of an error; confirm the object exists.

Restore rehearsal checklist

A backup you have never restored is a hypothesis. Rehearse in a disposable Rancher install that runs the same Rancher and rancher-backup versions.

• Install rancher-backup in the rehearsal environment and recreate the S3 and encryption secrets exactly.
• Inspect the restore schema with kubectl explain restore.spec; restore field names are also release-sensitive.
• Create a Restore CR that points to the backup object and storage location from the rehearsal bucket or folder.
• Track it with kubectl -n cattle-resources-system get restore <restore-name> -o yaml and wait for completion conditions, not just pod activity.
• Confirm the Rancher inventory returns: clusters, users, roles and project bindings.
• Before any production restore, take a fresh encrypted Backup of the current state so you have a rollback point. A restore can overwrite live Rancher state, so production recovery needs a maintenance window and a tested plan.

Limits: rancher-backup is not an etcd backup

Concern	Covered by rancher-backup?	Use a separate path
Rancher management-plane resources	Yes, via the selected ResourceSet	Verify the default selection in your release
Downstream RKE2/K3s etcd, the control-plane database	No	Scheduled etcd snapshots or cloud-provider control-plane backups
Application data and persistent volumes	No	Application and PV backup tooling
External IdP users and groups	No	Identity provider configuration or export process

Restores are version-sensitive. Restoring into a different Rancher version or a different cluster may fail. Losing the encryption secret or S3 credentials can make backups unusable even if the objects still exist.

Start small, then schedule

Begin with a one-time encrypted backup to object storage, verify the object and Backup status conditions, then convert to a scheduled Backup with retention. In parallel, confirm that downstream control planes have their own tested recovery path: RKE2/K3s scheduled etcd snapshots or cloud-provider control-plane backups. Rancher's front door is only as recoverable as the management state behind it.