Adopting Qodana Baselines for Incremental Code Quality in CI/CD

The problem: legacy violations block CI

When a team first runs a static analyzer on a mature codebase, the report often contains hundreds of findings. Failing the build on every one of them is impractical—developers need a path to improve quality without halting delivery. Qodana solves this with a baseline mechanism that records the current violation set, treats it as the accepted floor, and only flags new issues on subsequent runs.

How baselines work

Qodana writes a baseline file (either qodana.sarif.json or qodana.baseline.json) that contains every inspection result: rule ID, file path, line range, and a hash of the diagnostic message. The file is committed to version control. On each CI run Qodana loads the baseline, runs a fresh analysis, and produces a diff. Only violations not present in the baseline appear in the SARIF output, so the pipeline can fail on regressions while legacy debt stays silent.

Baselines are tied to the inspection profile and Qodana version. Changing the profile or upgrading Qodana can introduce new rules or alter rule IDs, which would make the old baseline report many false “new” violations. In that case you regenerate the baseline (see the worked example).

Worked example: adding a baseline to a GitHub Actions workflow

1. Generate the baseline locally (requires Docker and the Qodana CLI image). Run from the repository root:

   docker run --rm -v ${PWD}:/data \
     jetbrains/qodana-cli:2024.3 \
     qodana scan --baseline --show-report
   

   This creates qodana.sarif.json in the workspace. Commit it:

   git add qodana.sarif.json
    git commit -m "chore: add Qodana baseline"
    git push
       

2. Configure the CI job to use the baseline. In .github/workflows/qodana.yml:

   name: Qodana
   on: [push, pull_request]
   jobs:
     qodana:
       runs-on: ubuntu-latest
       permissions:
         contents: read
         security-events: write   # needed for SARIF upload
       steps:
         - uses: actions/checkout@v4
         - name: Run Qodana with baseline
           uses: JetBrains/qodana-action@v2024.3
           with:
             args: "--baseline"
         - name: Upload SARIF
           uses: github/codeql-action/upload-sarif@v3
           with:
             sarif_file: qodana.sarif.json
   

   The --baseline flag tells Qodana to load the committed baseline and only emit new findings.
3. Verify the behavior. Introduce a fresh violation (e.g., add an unused import) in a feature branch, open a PR, and watch the check fail with a single new SARIF entry. The baseline entries remain suppressed.

Trade‑offs and maintenance

• Line‑sensitivity: If a refactoring moves code lines, a suppressed violation may reappear as “new” because the baseline stores line numbers. Regenerate the baseline after large refactors (qodana scan --baseline --show-report) to avoid noise.
• Version upgrades: Upgrading Qodana or the underlying IntelliJ inspections can add rules or change IDs. Treat a major upgrade as a baseline reset point.
• Repository bloat: Legacy projects can produce multi‑megabyte SARIF files. Consider storing the baseline in Git LFS or an artifact store and referencing it from qodana.yaml via baseline: path/to/baseline.sarif.json.
• Hidden debt: Baseline‑only mode hides the total violation count from dashboards. Schedule periodic “baseline reset” sprints where the team fixes a slice of suppressed issues and regenerates the baseline, gradually lowering the floor.

Next steps: schedule baseline resets

Add a recurring calendar reminder (e.g., every sprint) to review the suppressed count in the SARIF file (jq '.runs[0].results | length' qodana.sarif.json). Pick a manageable subset, fix them, then rerun the baseline generation command. Over time the baseline shrinks, the quality gate tightens, and the codebase improves without ever breaking the build unexpectedly.