Adopting Qodana Incrementally with Baselines: An Architecture Note

The requirement: quality gates on a codebase that already fails them

You want static analysis in CI, but the first Qodana run on a mature repository reports thousands of pre-existing issues. Blocking merges on that report means nobody can ship until the backlog is fixed, which usually kills the adoption effort before it starts.

The useful takeaway: Qodana's baseline feature lets you snapshot the existing findings once, suppress exactly those findings in future runs, and enforce a "no new issues" gate from day one. The backlog becomes a separate workstream instead of a blocker.

Requirements

• Analysis must run in CI on every pull request without extra build infrastructure.
• Pre-existing issues must not fail the build, but must remain visible for tracking.
• Any genuinely new issue introduced by a change must fail the gate.
• The design must survive inspection-profile updates without silently re-hiding new problems.

The smallest suitable design

The design has three parts: a one-time baseline artifact, a CI step that consumes it, and a configuration file that pins behavior.

1. Generate the baseline once

Run Qodana locally against the current main branch with the baseline flag. From the repository root, using Docker (no local install required):

docker run --rm -v $(pwd):/data/project \
  jetbrains/qodana-jvm:2024.3 \
  --save-report --baseline /data/project/qodana.sarif.json

Wait — that flag consumes a baseline. To create one, run the analysis first and copy the generated SARIF report:

docker run --rm -v $(pwd):/data/project \
  -v $(pwd)/qodana-output:/data/results \
  jetbrains/qodana-jvm:2024.3

cp qodana-output/qodana.sarif.json qodana-baseline.sarif.json

Run these from a shell with Docker access; no elevated privileges are needed beyond Docker group membership. Pick the image matching your stack (qodana-jvm, qodana-python, qodana-js, etc.) and pin an explicit version tag rather than latest — image tag changes alter the inspection set and can break runs or invalidate the baseline. Commit qodana-baseline.sarif.json to the repository so CI and local runs share one source of truth.

2. Wire the baseline into CI

For GitHub Actions, the official action supports the baseline directly:

- uses: jetbrains/qodana-action@v2024.3
  with:
    args: --baseline,qodana-baseline.sarif.json

With a baseline in place, Qodana still reports everything, but findings that match the baseline are marked as covered by it and excluded from the failure count. New findings fail the step (non-zero exit code), which blocks the merge.

3. Pin configuration in qodana.yaml

version: "1.0"
profile:
  name: qodana.recommended
failureConditions:
  severityThresholds:
    any: 0

The failureConditions block makes the gate explicit: any new finding of any severity fails. Start there; loosen per-severity later if noise demands it. Validate the YAML before committing — a syntax error fails the analysis step with a configuration error, not a lint-style warning, and it will block every PR until fixed.

Trust and data boundaries

The baseline file is a trust boundary. Anyone who can modify it can suppress arbitrary findings, including ones they just introduced. Treat it like a lockfile:

• Require code-owner review for changes to qodana-baseline.sarif.json and qodana.yaml.
• Never regenerate the baseline inside the PR pipeline itself; generation must be a deliberate, reviewed local or scheduled job.
• The SARIF file contains file paths and code snippets from your repository. If you upload reports to Qodana Cloud or another dashboard, confirm that visibility matches your code's confidentiality.

Operational checks

• After the first baseline run, open the generated report (the HTML report in qodana-output, or the PR annotation view) and confirm baseline-matched findings are marked as such rather than counted as failures.
• Deliberately introduce a trivial issue (for example, an unused variable the profile flags) on a test branch and confirm the CI step fails. Remove it and confirm green. This is the only reliable proof the gate works.
• Check the Qodana step logs for the baseline summary line showing how many problems were matched against the baseline; a count of zero on a legacy codebase means the baseline is not being picked up (wrong path is the usual cause).
• Confirm flag availability for your pinned image with docker run --rm jetbrains/qodana-jvm:2024.3 analyze --help before relying on a flag in CI.

Failure modes

Baseline drift

The baseline is matched heuristically against findings. Two common drift causes:

• Inspection profile or linter version changes. Upgrading the Qodana image can rename inspections, change their messages, or add new ones. Old baseline entries stop matching (pre-existing issues resurface as "new" and fail the build), or — worse — a reworded inspection can let a genuinely new issue match a stale baseline entry. Mitigation: when you bump the image tag, regenerate the baseline in the same commit and review the diff.
• Code motion. Moving a file or heavily editing a function can change fingerprints enough that suppressed findings reappear as new. This is noisy but safe — it fails closed. The fix is a reviewed baseline refresh, not a looser gate.

Baseline as a landfill

If the baseline only grows, you have a suppression list, not a quality program. Track the baseline finding count as a metric and schedule burn-down work; some teams fail CI if the baseline file grows beyond a threshold size.

Silent misconfiguration

A wrong baseline path does not always produce an obvious error — the analysis may simply run without a baseline and fail on the full backlog, which looks like a Qodana bug. The log check above catches this.

Conditions that would change the design

• Greenfield project: skip the baseline entirely; enforce the full gate from the first commit.
• Small backlog (under ~50 findings): fixing the backlog outright is cheaper than operating a baseline; do that instead.
• Monorepo with heterogeneous modules: one global baseline becomes unwieldy. Move to per-module Qodana jobs with per-module baselines and profiles.
• Regulated environment requiring full audit trails: a suppression mechanism may need justification per finding; consider Qodana Cloud's per-issue review workflow instead of a flat SARIF file.

Limitations

Exact flag names, failure-condition syntax, and action inputs vary across Qodana releases; this note assumes the 2024.x line. Verify against the --help output of your pinned image and the action's README before committing the pipeline. Baseline matching is heuristic, so expect occasional false "new issue" failures after large refactors — treat them as a prompt for a reviewed baseline refresh, not a reason to disable the gate.