Speeding Up Bitbucket Pipelines with Dependency Caching

Use the caches section to reuse dependencies between runs

If your Bitbucket Pipelines build spends a lot of time downloading Maven, npm, or Composer dependencies, adding a caches block to bitbucket-pipelines.yml can restore those files automatically on each pipeline start. The cache is stored per‑repository, shared across branches, and is restored before the first step that needs it, then saved again after the step finishes.

Worked example: caching the Maven local repository

The following snippet shows a minimal configuration for a Java project that uses Maven. It caches the ~/.m2 directory, which holds downloaded jars and plugin artifacts.

image: maven:3.9.6

pipelines:
  default:
    - step:
        name: Build and test
        caches:
          - maven
        script:
          - mvn verify -B

# Define the cache
caches:
  maven: ~/.m2

When the pipeline runs:

1. Bitbucket looks for a cache named maven. If one exists, it downloads the archive and extracts ~/.m2 before the script step begins (you’ll see a “Restoring cache” line in the logs).
2. The mvn verify command runs, reusing any already‑downloaded dependencies.
3. After the step finishes, Bitbucket archives the contents of ~/.m2 and stores it as the cache for future runs (visible as a “Saving cache” line).

On the second run, the restore step is usually fast because only changed or new artifacts need to be downloaded, which can cut the dependency‑resolution phase from minutes to seconds.

Limits and constraints you should know

• Size limit: each cache cannot exceed 1 GB. If the directory grows larger, the save step will fail and the pipeline will be marked as failed.
• Expiration: a cache that has not been accessed for 7 days is automatically removed. Frequent runs keep it alive.
• Path specificity: only the exact paths you list are cached. Missing a sub‑directory (e.g., caching ~/.m2/repository but forgetting ~/.m2/plugin) leads to cache misses for those parts.
• Shared across branches: changing a dependency version in a feature branch does not bust the cache; the old version may be reused until the cache expires or you manually change the cache key.

Common mistakes and how to avoid them

• Incorrect path: typing ~/.m2 on Windows agents (which use a different home) results in an empty cache. Use the agent’s actual home directory or an absolute path like /root/.m2.
• Caching build outputs: storing generated JARs, WARs, or Docker images in the cache wastes space and can exceed the 1 GB limit. Keep the cache limited to dependency directories only.
• Forgetting to restore: if you place the caches list under a step but forget to include it, the pipeline will never restore the cache, and you’ll see no speed‑up. Verify the indentation under the step.
• Assuming automatic invalidation: updating pom.xml does not automatically create a new cache. If you need a fresh set of dependencies, either bump a cache key (e.g., maven-v2: ~/.m2) or clear the cache manually via Repository settings → Pipelines → Caches.

How to verify that caching is working

1. Run the pipeline once and note the total time, especially the duration of the mvn verify step.
2. In the pipeline logs, confirm you see lines similar to:

Restoring cache maven...
Saving cache maven...

1. Run the pipeline a second time without changing dependencies. Compare the logs: the restore step should finish quickly, and the build step should be noticeably shorter.
2. Optionally, temporarily comment out the caches block, run the pipeline again, and observe whether the build time increases, confirming the cache was previously active.

When caching may not help

If your project’s dependencies are already small (< 10 MB) or are fetched from a very fast internal proxy, the overhead of archiving and extracting the cache can outweigh the savings. In such cases, measure the pipeline duration with and without the cache to decide whether to keep it.