Choosing Between Bitbucket Pipelines Cache and Artifacts for Build Optimization

The Persistence Problem in Isolated Steps

In Bitbucket Pipelines, every step runs in a fresh Docker container. This isolation ensures a clean environment but means that any file created in Step A—such as a compiled binary or a downloaded library—is deleted before Step B begins. To move data across these boundaries, you must choose between caches and artifacts.

The decision depends on whether you are trying to speed up the build (via dependency reuse) or pass a required build output to a later stage (via explicit hand-off).

Comparing Cache and Artifacts

Feature	Cache	Artifacts
Primary Purpose	Speeding up subsequent runs (Dependencies)	Passing data between steps (Build Outputs)
Availability	Restored at start of step	Uploaded at end / Downloaded at start
Size Limit	1 GB per pipeline (Silent skip if exceeded)	Limited by runner disk space
Retention	Expires after 7 days of inactivity	Deleted when pipeline run completes
Scope	Across different pipeline runs	Within a single pipeline run

Trade-offs and Decision Logic

When to use Cache

Use caches for directories that are expensive to recreate but not strictly required for the build to succeed (e.g., node_modules, ~/.m2, ~/.cargo). The primary trade-off is speed versus reliability: caches are faster because they are restored transparently, but they can be evicted or expire, meaning your pipeline must be able to handle a "cache miss" by downloading everything from scratch.

When to use Artifacts

Use artifacts for files that are the direct result of a build step and are required for a subsequent step to function (e.g., a .jar file, a compiled binary, or test reports). The trade-off here is reliability versus overhead: artifacts are guaranteed to be there if the previous step succeeded, but they add I/O time to the pipeline because they must be explicitly uploaded and downloaded.

Implementation Example: Maven Build

This configuration demonstrates a hybrid approach: caching the Maven repository to save time on every run, while using artifacts to preserve test reports for auditing.

# bitbucket-pipelines.yml
image: maven:3.9.6-eclipse-temurin-17

pipelines:
  default:
    - step:
        name: Build and Test
        caches:
          - maven
        script:
          - mvn clean verify
        artifacts:
          - target/surefire-reports/**

# Define the custom cache path
caches:
  maven: ~/.m2

Configuration Details:

• Permissions: The pipeline runner requires write access to the defined cache path (usually handled by the image user).
• Placeholders: ~/.m2 is the standard Maven local repository path.
• Risk: If the target/surefire-reports/ directory is massive, it may slow down the transition to the next step.

Verification and Validation

To verify that your persistence strategy is working, check the following:

Checking Cache Hits

1. Trigger a pipeline run.
2. Open the step log. Look for the line: Restored from cache ~/.m2.
3. If you see No cache found for ~/.m2, it is either the first run, the cache expired after 7 days, or the cache exceeded the 1 GB limit.

Checking Artifact Availability

1. Once the pipeline completes, navigate to the Pipelines view.
2. Select the specific commit/run.
3. Click the Artifacts tab. You should see the files listed under target/surefire-reports/ available for download.

Limitations

• Silent Cache Failures: If your cache exceeds 1 GB, Bitbucket will not throw an error; it will simply skip saving the cache, leading to slower subsequent builds.
• Network Overhead: Large artifacts increase the total pipeline duration due to the upload/download cycle between the runner and the Bitbucket storage backend.
• Ephemeral Nature: Artifacts are not permanent. If you need to keep a build binary for long-term release, you must upload it to a dedicated artifact repository (like Nexus or Artifactory) rather than relying on pipeline artifacts.