Adopting Static Analysis Incrementally with Qodana Baselines and Quality Gates

The problem: static analysis on a codebase with history

You've decided to add static analysis to your CI pipeline. The first run produces 2,347 issues across your codebase. Most are pre-existing—style inconsistencies, deprecated API usage, nullable annotations missing from legacy modules. Fixing them all at once would stall feature work for weeks. You need a way to say "no new issues" without drowning in the backlog.

Qodana's baseline feature solves exactly this. It captures a snapshot of current inspection results, then subsequent runs report only what's new or worse. Combined with quality gates that fail the build on regressions, you can enforce standards on new code while paying down legacy debt on your own schedule.

How baselines capture the current state

A baseline is a JSON file (baseline.json) that records every inspection finding at a point in time: file, line, inspection ID, severity, and message. When you run Qodana with --baseline, it compares the fresh analysis against this snapshot. Findings that match the baseline (same file, line, inspection) are suppressed. Anything else—new files, new lines, new violations on changed lines—appears in the diff.

Generate it once, commit it, and the whole team shares the same reference point:

# In your repository root
docker run --rm -v $(pwd):/data/project \
  jetbrains/qodana-jvm:2024.3 \
  qodana scan --baseline --save-report

This produces qodana.sarif.json (full results) and baseline.json (the snapshot). Commit baseline.json to version control. On CI, run without --save-report to get baseline-aware output:

docker run --rm -v $(pwd):/data/project \
  jetbrains/qodana-jvm:2024.3 \
  qodana scan --baseline

The baseline works across all Qodana-supported languages (Java, Kotlin, JavaScript, TypeScript, Python, Go, PHP, etc.) because it uses the same inspection profiles as IntelliJ IDEA. Run the same Docker image locally and in CI—identical engine, identical results.

Quality gates: from "warn" to "fail" progressively

A baseline alone just filters noise. Quality gates turn the diff into policy. In qodana.yaml, configure thresholds that fail the build when crossed:

qualityGates:
  - name: "No new issues"
    condition: "newIssues > 0"
    failThreshold: 0
  - name: "Critical severity blocked"
    condition: "newIssuesBySeverity.CRITICAL > 0"
    failThreshold: 0
  - name: "Total issue budget"
    condition: "totalIssues > 5000"
    failThreshold: 5000

Start with failThreshold: 1 (warn only) or a generous budget. Tighten monthly. The gate evaluates new issues relative to the baseline, so legacy debt doesn't block merges—only regressions do.

For hotfixes that genuinely need to bypass the gate, Qodana supports an environment variable escape hatch:

QODANA_QUALITY_GATE_MODE=warn docker run ...

Use sparingly and audit afterward.

Worked example: baseline-aware PR annotations

Here's the end-to-end flow that lands baseline-aware results directly in GitHub pull requests.

1. Generate and commit the baseline (one-time, locally or in a bootstrap CI job):

   docker pull jetbrains/qodana-jvm:2024.3
   docker run --rm -v $(pwd):/data/project \
     jetbrains/qodana-jvm:2024.3 \
     qodana scan --baseline --save-report
   git add baseline.json
   git commit -m "chore: add Qodana baseline"
   git push

2. Add a GitHub Actions workflow (.github/workflows/qodana.yml):

   name: Qodana
   on:
     pull_request:
       types: [opened, synchronize, reopened]
   jobs:
     qodana:
       runs-on: ubuntu-latest
       permissions:
         security-events: write
         contents: read
       steps:
         - uses: actions/checkout@v4
           with:
             fetch-depth: 0
         - name: Run Qodana scan
           uses: JetBrains/qodana-action@v2024.3
           with:
             args: "--baseline"
         - name: Upload SARIF
           uses: github/codeql-action/upload-sarif@v3
           with:
             sarif_file: qodana.sarif.json

3. Open a PR with a new violation—for example, add a nullable parameter without @NotNull in a Kotlin file. The check runs, compares against baseline.json, and uploads only the new finding. In the PR's "Checks" tab you'll see the annotation on the exact line, with the inspection name (e.g., ConstantConditions) and a link to the rule description.

Developers see actionable feedback in context, not a wall of pre-existing noise.

Trade-offs and limitations to plan for

• Baseline size. Large codebases produce multi-megabyte baseline.json files. Commit them with Git LFS, or regenerate quarterly to avoid repository bloat.
• Profile or version changes invalidate baselines. Upgrading Qodana (e.g., 2024.3 → 2025.1) or switching inspection profiles re-runs inspections with potentially different logic. Regenerate the baseline after such changes and expect a one-time spike in "new" issues.
• Line sensitivity. Refactoring that moves code (extract method, reformat) shifts line numbers. The baseline matches by file+line+inspection, so moved code appears as "new" violations until you refresh the baseline with --baseline-include-absent:

  qodana scan --baseline --baseline-include-absent

  This absorbs intentional fixes and repositioned code in one step.
• Resource requirements. The Docker image needs 4–8 GB RAM and several GB disk for indexes. GitHub-hosted ubuntu-latest runners (7 GB RAM) usually suffice; self-hosted runners may need tuning.
• SARIF upload limits. GitHub caps SARIF at 10 MB per file, 25 MB per run. Very large diffs may need filtering (--include/--exclude patterns) or splitting by module.

Start small, tighten gradually

Run the baseline generation on a representative module first. Verify the diff behaves as expected: introduce a known violation, confirm it appears; fix it, confirm it disappears. Then add the quality gate with a warn-only threshold, watch a few PR cycles, and lower the threshold when the team is comfortable.

The baseline isn't a free pass on technical debt—it's a contract that new code meets a higher standard. Review the baseline quarterly: can you fix a chunk of suppressed issues and shrink the snapshot? That's the incremental path from "legacy codebase" to "clean build" without a big-bang rewrite.