Stop Fake Coverage Drops in Monorepo PRs with Codecov Carryforward Flags

A pull request that touches only the payments service should not fail coverage because the search service tests were skipped. In a monorepo with selective test execution, Codecov will show a coverage drop for every flag that does not receive a new upload on the PR. Carryforward flags solve this by reusing the last known report for an unchanged flag from the base commit.

The problem with selective testing and per-flag coverage

Monorepos often run tests per service. A CI job detects changed paths and runs only the relevant test suites. Each suite uploads coverage to Codecov with a flag name, e.g. payments or search. A flag is a label attached to a coverage report that lets Codecov group results.

Without carryforward, a PR that uploads only payments will show search as missing. The PR comment and status check can report a large overall drop even though search code was untouched. Teams either run all tests on every PR, which is slow and expensive, or they accept noisy coverage gates.

How carryforward flags work

Carryforward tells Codecov: if a flag has no upload on the head commit of a PR, reuse the report for that flag from the base commit. The reused report is marked as carried forward in the PR summary. Patch coverage is still computed from the changed files in the PR, and per-flag status checks can enforce thresholds only on the flags that were actually tested or carried forward.

Configuration lives in codecov.yml at the repository root. Flag definitions and the carryforward setting must match the flag names used in upload commands. Inconsistent naming causes uploads to be ignored and carryforward to fail silently.

flags:
  payments:
    carryforward: true
  search:
    carryforward: true

coverage:
  status:
    project:
      default:
        target: auto
    patch:
      default:
        target: 80%

The carryforward: true setting applies to the flag for pull requests. Status checks can be scoped per flag in Codecov project settings to require a threshold only for the flags relevant to the change.

A concrete setup for a monorepo

CI runs a changed-paths detector and starts a test job per affected service. The job uploads coverage with a matching flag name only when the service changes.

Example upload step run in the CI job for a service, typically in the job runner with access to the Codecov token as a secret:

bash -c "codecov -t $CODECOV_TOKEN -f coverage.xml -F payments"

Placeholders: $CODECOV_TOKEN is a repository secret with upload permission, coverage.xml is the path to the report produced by the test runner, and -F payments is the flag name that must exactly match the name in codecov.yml.

Expected checks after merge to main: the Codecov PR comment lists coverage per flag. Flags with a fresh upload show a new percentage, flags without an upload show a carried-forward value with an indicator that it was taken from the base commit. Status checks reflect both carried-forward and fresh reports.

Risk: if the flag name in the upload command differs in case or spelling from the definition in codecov.yml, Codecov treats it as an unknown flag. The report is stored but carryforward will not apply, leading to missing coverage in the PR summary.

Trade-offs and limitations

Carried-forward data is stale by definition. It will not reflect changes made to the base branch after the last upload for that flag. A regression introduced on main can be masked until the flag is re-tested.

Relying on carryforward can hide how long a service has gone without a real test run. A policy for periodic full runs, e.g. nightly or on a schedule, helps refresh the baseline.

Flag granularity matters. Overly fine flags make overall project coverage hard to interpret and increase configuration surface. Coarse flags reduce selectivity.

Practical verification without assuming live data:

• Inspect a PR on a monorepo using Codecov and confirm the PR comment lists coverage per flag with an indicator for carried-forward flags.
• Review codecov.yml for flag definitions and carryforward settings and compare flag names with the names used in CI upload commands.
• Create a test PR that changes only one service, verify that only that flag shows a new coverage upload while other flags show carried-forward values in the report summary.
• Check Codecov project settings for status check configuration to see which flags are required and what thresholds are enforced for patch and total coverage.

Actionable closing

Use carryforward flags when you want per-service coverage gating without running every test suite on every PR. Keep flag names consistent between CI uploads and codecov.yml, scope status checks to the flags that matter for the change, and add a periodic full test run to refresh stale baselines. That gives fast PR feedback and avoids artificial coverage drops for untouched services.