Freeze Legacy Debt with Qodana Baselines and Quality Gates

The wall of errors that blocks merges

Running Qodana for the first time on a mature codebase often surfaces hundreds of existing violations. Enabling a strict gate on total issues freezes development. Ignoring the results makes the tool noise. The practical engineering decision is to freeze known legacy debt and gate only on new issues introduced by a change.

Qodana supports this with baselines and quality gates. A baseline records the current set of violations so subsequent runs can compare against it. A quality gate evaluates the scan result against thresholds and can fail a CI step. Used together, they let a team say: we will not add new problems, even if the codebase already contains old ones.

What a baseline does in practice

A baseline is a JSON file, commonly named qodana.baseline.json, that captures issue identifiers from a scan. Subsequent scans compare results to that file and report which issues are new. The baseline can be committed to the repository and reviewed in PRs when intentionally updated.

Baseline handling differs between Qodana Cloud and Qodana On-Prem. Cloud stores baselines per project in the UI; On-Prem relies on file system or artifact storage. CI scripts must reference the correct location for your deployment. Exact file schema and fingerprinting details are version dependent and should be verified against your release documentation.

Incremental quality gate in CI

Quality gate configuration lives in qodana.yaml under qualityGates:. Model knowledge describes options such as failThreshold and newIssuesThreshold, with per-category overrides for license audit, duplicates and security. The exact keys and semantics vary by version, so check the versioned schema before use.

A common pattern described in current knowledge is to run PR pipelines with a baseline and a fail threshold of zero for new issues, while scheduled main-branch jobs run with a baseline but without a hard fail to track trends. The behavior of --fail-threshold together with --baseline is version sensitive; verify the exit code semantics in your Qodana release.

Example PR check with placeholders

Run on a Linux CI agent with permission to mount the workspace. Replace <language> and <version> with values supported in your environment. Ensure the baseline file is present in the workspace; if it is missing, the run behaves like a first run.

# Example command shape, not tested
# Permissions: runner can read/write workspace and run Docker
# Risk: generated or vendored code can create new issues unless excluded in qodana.yaml

docker run --rm \
  -v $(pwd):/data \
  jetbrains/qodana-<language>:<version> \
  scan \
  --baseline qodana.baseline.json \
  --fail-threshold 0

With this shape, the step is intended to return a non-zero exit code when the scan finds issues not recorded in the baseline. If the PR only touches files with pre-existing baseline issues and adds no new findings, the step is intended to pass. Actual exit behavior must be confirmed in your version.

Managing the baseline lifecycle

A baseline should be updated intentionally. When legacy issues are fixed, the baseline becomes stale and will start reporting those fixed issues as new on the next run. A typical approach is to regenerate the baseline after a cleanup batch in a dedicated commit, so the change is reviewable.

Keep two scan profiles:

Pipeline	Typical intent	Purpose
PR / Merge Request	baseline + threshold	Prevent new regressions
Scheduled main branch	baseline, no hard fail	Track total debt trend and find low effort fixes

Exclusions for generated code and vendored dependencies should be defined in qodana.yaml via inspections.exclude or excludePaths to avoid noise in the new-issue count.

Limitations and trade-offs

Reported cautions from model knowledge include:

• Baseline fingerprints can be sensitive to large refactors or auto-formatting. A global format change may cause many existing issues to appear as new. Plan baseline updates after formatting changes.
• --fail-threshold 0 counts all new issues unless explicitly excluded. Generated code and dependencies can trigger failures.
• Baselines hide the scale of legacy debt. Without visible scheduled reports, technical debt repayment can deprioritize.
• Docker file system latency on macOS and Windows can increase scan time versus Linux runners. Prefer Linux CI agents for consistent timing.

SARIF export --format sarif is described as useful for visualization in GitHub Code Scanning or GitLab SAST UI. Quality gate pass/fail logic runs inside the Qodana step itself; SARIF is for visualization, not gating.

Actionable checks

To verify your setup without assuming behavior:

1. Run a scan with an existing baseline and no code changes. Record the exit code and the new-issue count reported in the HTML report.
2. Introduce a deliberate, easily identifiable violation in a test branch and run the same command. Check whether the exit code changes and whether the issue appears as new versus baseline.
3. Fix a known baseline issue, regenerate the baseline in a separate commit, and compare the new-issue count in the scheduled report to confirm the baseline moved.

Always validate qodana.yaml keys against the versioned documentation for your Qodana release, and treat baseline updates as code reviewable changes.