Qodana Baseline Drift Diagnosis

When Qodana reports “New problems found” (exit 253) and the CI pipeline fails, the most common culprit is a stale or mismatched baseline. This guide walks through the recognizable symptoms, diagnostic steps, and concrete fixes to get your Qodana checks back on track.

Problem: CI fails with exit code 253 after a Qodana scan

In a typical workflow the Qodana container finishes with one of three exit codes:

0 – clean, no new issues
253 – new problems found compared to the baseline
1 – error during analysis

If you see 253 and the pipeline aborts, you’re most likely dealing with a baseline drift: the SARIF file that represents the accepted issues no longer matches the current inspection profile or the repository state.

Recognizable Condition

CI logs show a line like:

qodana: exit code 253 – New problems found

and the Qodana report is uploaded to GitHub Code Scanning or a similar system. The baseline file qodana.sarif.json is present in the repository but the number of issues has increased.

Cause / Diagnostic Table

Possible Cause	Diagnostic Indicator
Baseline file missing or stale	Baseline path in `qodana.yaml` wrong or file not committed.
Inspection profile changed	New inspections enabled in `qodana.yaml` not reflected in baseline.
License token expired/invalid	CI logs contain “License validation failed” or `QODANA_TOKEN` missing.
Memory exhaustion	Logs show “OutOfMemoryError” or “GC overhead limit exceeded” and the SARIF file is smaller than expected.

Ordered Checks

Verify baseline path and inspection profile in qodana.yaml
```
cat qodana.yaml
```
Ensure baseline: qodana.sarif.json points to the correct file and the profile matches the one used in CI.
Run a local scan with baseline comparison
```
docker run --rm -v $(pwd):/data -e QODANA_TOKEN jetbrains/qodana-jvm:2024.3.1 qodana scan --baseline
```
Check the exit code: 0 means the local scan matches the baseline.
Compare SARIF issue counts
```
jq '.runs[0].results | length' qodana.sarif.json
```
Run the same command after a fresh scan and compare the counts. A discrepancy indicates drift.
Inspect CI logs for licensing errors
Search for “License validation failed” or “account.jetbrains.com” errors. Also verify the QODANA_TOKEN environment variable is set in the CI job definition.
Check memory usage
Look for JVM errors in the logs. If present, increase the heap with QODANA_JVM_OPTIONS=-Xmx4g or enable incremental analysis.

Fixes Tied to Findings

Stale baseline – Regenerate the baseline after reviewing the new issues:
```
docker run --rm -v $(pwd):/data -e QODANA_TOKEN jetbrains/qodana-jvm:2024.3.1 qodana scan --baseline --show-report
```
Commit the updated qodana.sarif.json after manual review.
Profile drift – Update qodana.yaml to match the baseline or regenerate the baseline with the new profile. Use --baseline-include or --baseline-exclude patterns to fine‑tune.
Token issues – Refresh the token in JetBrains account, store the new value as a CI secret named QODANA_TOKEN, and verify connectivity with:
```
curl -H "Authorization: Bearer $QODANA_TOKEN" https://account.jetbrains.com/licenses
```
Expect a 200 response with an active license.
Memory exhaustion – Add QODANA_JVM_OPTIONS='-Xmx4g' to the CI job or enable incremental analysis with --incremental. Ensure the CI cache key includes the hash of qodana.yaml so the incremental cache stays in sync.

Escalation Criteria

Baseline regeneration fails with internal errors after multiple attempts.
License validation passes locally but CI cannot reach account.jetbrains.com for >10 min.
OutOfMemoryError persists even after raising the heap to 8 GB and increasing timeout limits.
Generated SARIF does not conform to the 2.1.0 schema, blocking upload to GitHub Code Scanning.

Practical Verification Checklist

Run the local scan command and confirm exit code 0.
Verify SARIF issue count matches the baseline count.
Check CI cache hit rate for the qodana-cache key.
Run curl against the license endpoint to ensure the token is valid.
Review the scan logs for any JVM errors; absence of such errors indicates sufficient memory.

Limitations & Version Notes

Baseline files are tightly coupled to the inspection profile. Switching from qodana.recommended to qodana.starter invalidates the existing baseline without a warning.
Qodana 2024.3+ defaults to SARIF 2.1.0. Older CI actions that expect 2.0.0 may fail to upload the report.
Air‑gapped CI environments require an Enterprise offline license; otherwise the token validation will fail.
Monorepos with multiple Gradle modules may need --project-dir flags to target each module; otherwise Qodana may skip files.

Conclusion

Baseline drift is the most frequent cause of a Qodana pipeline abort. By following the ordered checks above you can quickly pinpoint whether the issue lies in the baseline file, the inspection profile, licensing, or resource limits. Once the root cause is identified, applying the corresponding fix will restore a clean exit code and keep your static‑analysis results reliable.

Diagnosing Qodana Baseline Drift in CI Pipelines