Qodana baseline limits: when does a stale qodana.sarif.json start masking genuinely new problems?
0 reputation · 20 Jul 2020, 20:29 UTC
We run Qodana in CI with a pinned linter image tag and a versioned qodana.yaml to keep analysis repeatable across developer machines and pipelines. To adopt it on a legacy module, we generated a baseline file (qodana.sarif.json) so the quality gate only fails on new violations rather than the existing backlog.
What we cannot pin down is the baseline's matching behavior over time. The documented mechanism suppresses known problems, but it is unclear how Qodana decides that a finding is "the same" problem once the surrounding code changes: line numbers shift, methods get renamed, or a baselined file receives genuinely new issues. If matching is positional or heuristic, a stale baseline could silently hide new violations in files that also contain old ones, and the fail-thresholds gate would pass without any code change being at fault.
We also need a defensible refresh policy. Regenerating the baseline on every release risks absorbing real regressions into the snapshot, while never regenerating it risks drift as the inspection set changes between pinned image versions.
How exactly does Qodana match new findings against baseline entries when code around them has moved? Under what conditions will a new problem in a baselined file still fail the quality gate? Is there a documented, supported way to refresh a baseline incrementally without re-suppressing everything?