What approaches can be used to validate that a CircleCI backup fully restores a project’s configuration and data integrity?

Question

Validating a CircleCI backup requires confirming that the exported archive contains every element necessary to rebuild the project’s configuration, workflows, and associated artifacts, while also preserving any runtime‑generated values such as environment variables or dynamic parameters.

Because the restoration must be tested without impacting the primary pipeline, an isolated environment is needed, and the comparison must rely on static analysis of the configuration files and deterministic artifact checks rather than on executing the restored workflows in production.

What techniques exist to verify that all configuration keys, including those set via context or parameters, are present in the backup? How can one compare the workflow graph of the restored project to the original without triggering a build? Which artifact comparison methods provide sufficient assurance of data integrity while respecting the isolation constraint?

Tasadduq Burney · Answer

Answer the exact question first
To prove that a CircleCI backup fully restores a project, you need a two‑phase audit:

Static configuration audit – ensure every YAML element, context, and environment variable is present and unchanged.
Deterministic runtime audit – run a repeatable pipeline that produces known artifacts and logs, then compare the outputs with the original environment.

Static Configuration Audit

Export the original project configuration:
curl -s -H "Circle-Token: $TOKEN" \
  https://circleci.com/api/v2/project/github/$ORG/$REPO/config | jq .config | tee original-config.yaml

Hash the YAML (ignoring non‑deterministic fields such as timestamps):
sha256sum original-config.yaml > original.sha

After restore, repeat the export and hash:
curl -s -H "Circle-Token: $TOKEN" \
  https://circleci.com/api/v2/project/github/$ORG/$REPO/config | jq .config | tee restored-config.yaml
sha256sum restored-config.yaml > restored.sha

Compare checksums:
diff original.sha restored.sha

If the files differ, investigate added or missing keys. The jq .config step removes the metadata block that CircleCI injects on export, keeping the comparison deterministic.
Validate the YAML syntax and schema with CircleCI’s validation endpoint:
curl -X POST -H "Content-Type: application/json" \
  -d @restored-config.yaml \
  https://circleci.com/api/v2/project/github/$ORG/$REPO/config/validate

The response must be {"valid":true}.

Context and Environment Variable Verification

List contexts:
circleci contexts list --org $ORG > contexts.txt

Export each context’s variables and compute a checksum per context.
After restore, repeat the process and compare each context’s checksum. A mismatch indicates a missing or altered variable.

Deterministic Runtime Audit
Run a small, idempotent job that generates a predictable artifact (e.g., a checksum of a static file). Use the same job in both environments.

Define a job in .circleci/config.yml:
jobs:
  validate:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run: echo "foo" > /tmp/foo.txt
      - run: sha256sum /tmp/foo.txt > /tmp/foo.sha
      - store_artifacts:
          path: /tmp/foo.sha

Trigger the job manually:
circleci run --job validate

Download the artifact from the original repo and compute its SHA256:
curl -s -H "Circle-Token: $TOKEN" \
  https://circleci.com/api/v2/project/github/$ORG/$REPO/artifact/1 | sha256sum

Repeat the same steps in the restored environment and compare the SHA256 values. Identical values confirm that the job ran in the same context and that artifacts were preserved.

Run History Cross‑Check
Use the runs endpoint to verify that the restored project has the same number of completed runs and that no runs are missing.
curl -s -H "Circle-Token: $TOKEN" \
  https://circleci.com/api/v2/project/github/$ORG/$REPO/runs | jq '.items | length'

Compare the counts from the original and restored projects.

External Artifacts and Data Stores
If your backup includes external objects (S3, Docker images), download the same object from both environments and compare file sizes and checksums. For Docker images, use docker pull and docker inspect --format='{{.Id}}' to confirm identical layers.

Assumptions and Caveats

Both backup and restore occur under the same CircleCI plan; feature differences can cause hidden mismatches.
API tokens used during export and import must have identical permissions.
Non‑deterministic fields (timestamps, run IDs) are stripped before checksum comparison.
Large artifacts may be truncated if the backup size limit is reached; verify that the backup settings allow full artifact retention.

Ask for missing diagnostic detail
To tailor the validation further, it would help to know whether your project uses dynamic contexts or runtime‑generated environment variables that are not captured in the static config export.

What approaches can be used to validate that a CircleCI backup fully restores a project’s configuration and data integrity?

1 answer

Answer the exact question first

Static Configuration Audit

Context and Environment Variable Verification

Deterministic Runtime Audit

Run History Cross‑Check

External Artifacts and Data Stores

Assumptions and Caveats

Ask for missing diagnostic detail

Your perspective belongs here.

1 question comment