Exporting and Importing BuildKit Cache to Accelerate Reproducible Docker Builds

Why Export/Import Cache Matters

When a CI pipeline builds the same image multiple times, the majority of the time is spent re‑executing unchanged steps. Docker BuildKit can skip those steps if it has a compatible cache available. By exporting that cache once and reusing it in subsequent builds, you reduce build times, lower bandwidth usage, and increase reproducibility. The core idea is to treat the cache as a first‑class artifact that can be stored locally or in a registry.

Getting Started: Enable BuildKit and Export the Cache

BuildKit is the modern build engine shipped with Docker 18.09+. To use the --export-cache flag you must enable it explicitly. The simplest approach is to set the environment variable DOCKER_BUILDKIT=1 or add a # syntax = docker/dockerfile:experimental line at the top of your Dockerfile. The following example demonstrates a complete workflow that exports the cache to a local directory.

# Dockerfile
# syntax = docker/dockerfile:experimental
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

Export the cache during the first build:

DOCKER_BUILDKIT=1 docker build \
  --build-arg BUILDKIT_INLINE_CACHE=1 \
  --export-cache type=local,dest=./buildkit-cache \
  -t myapp:latest .

Key points:

• --build-arg BUILDKIT_INLINE_CACHE=1 tells BuildKit to embed a cache‑metadata layer so the exported cache can be reused later.
• --export-cache type=local,dest=./buildkit-cache creates a directory containing a cache.tar file and a metadata.json descriptor.
• Run this command once; subsequent builds can import the cache.

Reusing the Cache in a New Build

With the cache available, a new build can reference it via --cache-from. BuildKit will automatically match cache entries to the current Dockerfile and context; if either changes, the corresponding step will be rebuilt.

DOCKER_BUILDKIT=1 docker build \
  --cache-from=./buildkit-cache \
  -t myapp:latest .

When you run this, the output will show Using cache for steps that match the exported cache. You can verify the speedup by comparing the elapsed time with a fresh build.

Storing the Cache in a Registry

For distributed teams, pushing the cache to a registry keeps it portable. BuildKit supports type=registry export:

DOCKER_BUILDKIT=1 docker build \
  --build-arg BUILDKIT_INLINE_CACHE=1 \
  --export-cache type=registry,ref=myrepo/cache:latest \
  -t myapp:latest .

To import from the registry:

DOCKER_BUILDKIT=1 docker build \
  --cache-from=myrepo/cache:latest \
  -t myapp:latest .

Because the registry holds a tarball of the cache, the same cache can be shared across multiple CI workers or even across different projects that share the same base layers.

How BuildKit Tags Cache Entries

Each cache entry is keyed by a hash of the Dockerfile and the build context. If you modify any instruction or add a file to the context, BuildKit invalidates the corresponding cache entry. This guarantees that you never reuse a stale layer that no longer reflects the source.

Common Mistakes and How to Avoid Them

• Forgetting to enable BuildKit: Without DOCKER_BUILDKIT=1 or the experimental syntax line, the --export-cache flag is ignored.
• Missing BUILDKIT_INLINE_CACHE: The exported cache will not contain the inline metadata needed for reuse, leading to cache misses.
• Exporting a cache that contains sensitive data: The build context is zipped into the cache. If the context includes secrets, those will be exposed when the cache is shared.
• Using a public registry for the cache: Intermediate layers can be inspected by anyone who pulls the cache image. Use a private registry or prune secrets before export.
• Assuming cache is immutable: The cache can be updated by re‑exporting after a new build. Always keep the cache in sync with the latest Dockerfile.

Checking That the Cache Works

After exporting, inspect the metadata.json file in the local cache directory. It contains a layers array with digests and build steps. When you run a subsequent build with --cache-from, the log should show “Using cache” for each matched step. If you modify a file that triggers a rebuild, the log will instead show “Rebuilding” for that step, confirming that the hash invalidation works as expected.

When to Use Cache Export/Import

• CI pipelines that build the same image repeatedly.
• Teams that share a common base image across multiple services.
• Environments with limited bandwidth where re‑downloading layers is costly.
• Projects that require reproducible builds; the cache ensures deterministic layer ordering.

Final Takeaway

Exporting and importing BuildKit cache is a lightweight, first‑class strategy to accelerate Docker builds and enforce reproducibility. By enabling BuildKit, exporting the cache once, and reusing it via --cache-from, you can cut CI build times dramatically while keeping the process secure and deterministic.