Separating Coverage Thresholds per Test Suite in Codecov Using Flags and Carryforward

Separating Coverage Thresholds per Test Suite in Codecov Using Flags and Carryforward

When a monorepo runs multiple test suites in parallel CI jobs, each upload can be tagged with a flag. The flag lets you set its own coverage threshold and view a separate badge, while carryforward merges the results so the final PR check reflects the combined coverage.

Flag definition

A flag is a named category that the upload CLI associates with a set of paths. In codecov.yml you declare the flag and the glob patterns that should be auto‑assigned.

flags:
  unit:
    paths: ["packages/*/src"]
  integration:
    paths: ["packages/*/src"]

The key name after flags: must match exactly the -F argument you pass to the codecov command.

Carryforward

Carryforward keeps coverage data from earlier jobs in the same commit SHA, allowing a later job to add its numbers to the existing total. Enable it with carryforward: true and optionally prune stale files with prune: true.

carryforward: true
prune: true

Worked example

In a monorepo, job A runs unit tests and uploads with -F unit. Job B runs integration tests and uploads with -F integration. The shared codecov.yml defines the flags and thresholds:

# codecov.yml
flags:
  unit:
    paths: ["packages/*/src"]
    threshold: 80%
  integration:
    paths: ["packages/*/src"]
    threshold: 60%
carryforward: true
prune: true

When the PR is checked, Codecov shows two rows — one for unit and one for integration — each evaluated against its own threshold after the carryforward merge.

Limits and common mistakes

• Carryforward only works within the same commit SHA; force‑pushes or rebases reset the window.
• Flag names are case‑sensitive; -F Unit will not match a unit key.
• Path globs follow GitHub Actions syntax; ** matches across directories, * does not.
• Thresholds are evaluated after the merge, so a late‑running job with low coverage can cause a flag failure even if earlier jobs passed.
• If prune: true is set and a job touches no new files, its coverage may be dropped from the final result.
• Self‑hosted Codecov Enterprise may lag behind SaaS in flag and carryforward support; verify your version.

Verification steps

1. Create a test repository with two GitHub Actions jobs that upload coverage using -F unit and -F integration.
2. Add the example codecov.yml above.
3. Open a pull request and confirm the Codecov check displays separate flag rows with the correct thresholds.
4. Inspect the Codecov dashboard “Flags” tab to see per‑flag coverage history and the merged values.
5. Run codecov --version in the CI to ensure the CLI version is 3.0.0 or newer, which supports carryforward.
6. Check the PR comment; if you set comment.layout: "diff" and behavior: "new", the comment appears only when coverage changes.

Practical check

After the verification steps, open the Codecov UI, navigate to the “Coverage” tab, and verify that the badge for each flag reflects the combined percentage after carryforward. If the badge shows a lower value than expected, re‑run the jobs with codecov --verbose to see how each upload is categorized.