Reducing Monorepo CI Costs with CircleCI Dynamic Configuration

The Monorepo Bottleneck

In a monorepo, a single commit to a documentation file or a small utility library often triggers the entire CI suite. When your repository houses multiple services, APIs, and frontend apps, running every test and build for every commit is a waste of compute credits and developer time. The goal is to ensure that only the code affected by a change—and the components that depend on it—actually run in the pipeline.

The Setup-Workflow Pattern

CircleCI solves this through Dynamic Configuration. Instead of a static .circleci/config.yml that defines every possible job, you use a two-stage process. First, a lightweight "setup" workflow runs. This workflow doesn't perform builds; it evaluates the commit and generates a new, tailored configuration file on the fly to be used by the rest of the pipeline.

This is achieved using the continuation orb. The setup job determines which parameters to pass to the next stage, effectively "continuing" the pipeline with a specific set of instructions based on the current git state.

Implementing Path Filtering

The path-filtering orb automates the logic of mapping file changes to pipeline parameters. You define a set of glob patterns (e.g., services/api/**) and associate them with boolean parameters. If any file matching the pattern changes, the parameter is set to true.

Example Configuration

To implement this, you need two files: the entry point .circleci/config.yml and the continued configuration .circleci/continue_config.yml.

1. The Entry Point (.circleci/config.yml)
Run this in your project root. This config must be minimal to keep the setup phase fast.

version: 2.1

setup: true

orbs:
  continuation: circleci/continuation@1.0
  path-filtering: circleci/path-filtering@1.0

workflows:
  setup-workflow:
    jobs:
      - path-filtering/filter:
          base-revision: main
          glob: 
            - 'services/api/**/*' : api-changed
            - 'services/web/**/*' : web-changed
          mapping-strategy: replace

2. The Continued Config (.circleci/continue_config.yml)
This file defines the actual work. It uses the parameters set by the filtering orb to decide which jobs to execute.

version: 2.1

parameters:
  api-changed:
    type: boolean
    default: false
  web-changed:
    type: boolean
    default: false

jobs:
  build-api:
    docker:
      - image: cimg/node:18.0
    steps:
      - checkout
      - run: npm run build-api

  build-web:
    docker:
      - image: cimg/node:18.0
    steps:
      - checkout
      - run: npm run build-web

workflows:
  main-workflow:
    jobs:
      - build-api: 
          when: "\"${{ .Env.CIRCLE_PROJECT_BRANCH }}\" == \"main\" || parameters.api-changed"
      - build-web:
          when: "\"${{ .Env.CIRCLE_PROJECT_BRANCH }}\" == \"main\" || parameters.web-changed"

Verification and Diagnostics

To verify the setup is working, push a commit that only modifies files within services/api/. In the CircleCI dashboard, you should see the "Setup" job complete, followed by the build-api job. The build-web job should not appear in the graph at all.

If a pipeline behaves unexpectedly, check the Setup job logs. The logs will explicitly print the resolved parameters and the final configuration JSON sent to the CircleCI API. This is the fastest way to debug whether a glob pattern is failing to match your file structure.

Trade-offs and Limitations

Dynamic configuration introduces a small amount of overhead—the setup job typically takes a few seconds to run. More importantly, there are two critical risks to manage:

• Branch Protection: If your GitHub/GitLab settings require a specific status check (e.g., build-web) to pass before merging, but the path-filter skips that job, the PR will be blocked indefinitely because the check never reports a status. To fix this, you must either make the check optional or ensure the job always runs on the target branch.
• Base Revision Accuracy: The orb relies on comparing the current commit to a base-revision. In cases of force-pushes or very shallow clones, the orb may fail to detect changes correctly, potentially skipping necessary builds or triggering everything.

Actionable Summary

If your monorepo CI is slowing down, move away from a single static config. Implement the path-filtering orb to map directories to boolean parameters, and use the when clause in your continued config to gate your jobs. Start by mapping your largest, slowest services first to see the most immediate impact on your credit consumption.