Speed up Bitbucket Pipelines with parallel steps

Problem: sequential steps waste time

When a Bitbucket Pipelines build compiles code, runs unit tests, and then builds a Docker image, each step waits for the previous one to finish. If the tasks are independent, the pipeline spends minutes idle while one step runs and the others sit queued, inflating feedback loops for developers.

Thesis: using the parallel: block lets independent steps run side‑by‑side, cutting total pipeline duration without changing the underlying tools.

How parallel steps work

In bitbucket-pipelines.yml a step can contain a parallel: key whose value is a list of sub‑steps. Each sub‑step gets its own executor, so they start at the same time and run until the slowest one finishes. The stage is considered complete only when all parallel sub‑steps have succeeded.

Worked example: Maven compile and test in parallel

Assume a Java project where mvn compile and mvn test do not share mutable state. The following configuration runs them concurrently:

# bitbucket-pipelines.yml
image: maven:3.9.0

pipelines:
  default:
    - step:
        name: Build and test
        caches:
          - maven
        parallel:
          - step:
              name: Compile
              script:
                - mvn compile -DskipTests
          - step:
              name: Test
              script:
                - mvn test

Both sub‑steps inherit the maven cache, so dependencies are downloaded only once. The total wall‑clock time is roughly the duration of the slower sub‑step rather than the sum of both.

Verification steps

1. Create a test repository and add the bitbucket-pipelines.yml above.
2. Push a commit to the default branch.
3. In the Pipelines UI, open the latest run and look for the Build and test step.
4. Expand it; you should see two child steps labeled Compile and Test with overlapping timestamps in their logs.
5. Compare the pipeline duration with a version where the two script commands are placed sequentially in a single step; the parallel version should show a noticeable reduction.

Trade‑off: concurrency limits

Bitbucket Cloud enforces a maximum number of concurrent pipelines per plan (Free: 2, Standard: 5, etc.). If you define more parallel sub‑steps than your plan allows, the excess pipelines are queued, which can erase the speed‑up. Monitor usage under Settings → Pipelines usage and adjust the number of parallel steps or upgrade the plan if you regularly hit the ceiling.

Limitation: shared state must be explicit

Parallel steps do not automatically share files or environment variables. If both steps need the same artifact, you must either restore it from a cache or pass it via the artifacts: mechanism; otherwise each step will rebuild or re‑download the same data, wasting time and possibly causing inconsistencies.

Actionable closing

Start small: pick a stage where two independent tasks exist (e.g., linting and unit testing, or compiling two separate modules). Add a parallel: block, verify overlapping logs, and measure the gain. If you see queues, reduce the parallelism or consider a plan upgrade. Over time, you can refactor more stages to run in parallel, trimming feedback cycles and letting developers get faster results.