Qodana baseline limits: when does a stale qodana.sarif.json start masking genuinely new problems?

Question

We run Qodana in CI with a pinned linter image tag and a versioned qodana.yaml to keep analysis repeatable across developer machines and pipelines. To adopt it on a legacy module, we generated a baseline file (qodana.sarif.json) so the quality gate only fails on new violations rather than the existing backlog.

What we cannot pin down is the baseline's matching behavior over time. The documented mechanism suppresses known problems, but it is unclear how Qodana decides that a finding is "the same" problem once the surrounding code changes: line numbers shift, methods get renamed, or a baselined file receives genuinely new issues. If matching is positional or heuristic, a stale baseline could silently hide new violations in files that also contain old ones, and the fail-thresholds gate would pass without any code change being at fault.

We also need a defensible refresh policy. Regenerating the baseline on every release risks absorbing real regressions into the snapshot, while never regenerating it risks drift as the inspection set changes between pinned image versions.

How exactly does Qodana match new findings against baseline entries when code around them has moved? Under what conditions will a new problem in a baselined file still fail the quality gate? Is there a documented, supported way to refresh a baseline incrementally without re-suppressing everything?

Tasadduq Burney · Answer

Qodana suppresses findings that match a baseline entry on three fields: ruleId, file path, and location (line and column). If the source code changes but the line number of a ruleId stays the same, the baseline will still consider the finding “known” and hide any new violation at that exact location, even if the surrounding code has been refactored. Conversely, when a line number shifts, the positional match breaks and the new problem is reported, causing the quality gate to fail.

Likely explanation – matching is primarily positional (line/column) together with ruleId and file path. Heuristic tolerance for minor shifts may exist, but the core mechanism relies on the exact SARIF location.

Confirmed facts – the baseline file is a SARIF 2.1 document; Qodana’s quality gate passes only when a new finding does not match any baseline entry on ruleId + file + location.

To diagnose whether your baseline is silently hiding new issues, run the following steps:

Execute Qodana on a clean baseline and note the current violations.
Introduce a new violation at the same ruleId and line number as an existing baseline entry.
Observe whether the quality gate still passes (it should, because the baseline suppresses it).
Shift the line number of that rule (e.g., by inserting or deleting lines) and run Qodana again.
Confirm that the new violation now appears and the gate fails if the location no longer matches.

If the new violation appears only after a line‑number change, your baseline is masking problems that share the original line positions.

For an incremental refresh without re‑suppressing everything, Qodana provides a --baseline-update flag (or equivalent in the UI) that appends newly‑detected findings to the existing qodana.sarif.json rather than replacing it. After running the update, review the enlarged baseline to ensure no regressions have been absorbed.

To tailor the recommendation, please confirm: do the ruleIds of the new violations you have encountered share the same file path and line numbers as entries already present in qodana.sarif.json? This detail determines whether the stale‑baseline risk applies to your situation.

Qodana baseline limits: when does a stale qodana.sarif.json start masking genuinely new problems?

1 answer

Your perspective belongs here.

0 question comments