Using Qodana Baselines to Hide Legacy Issues While Catching New Ones

The problem: legacy noise drowns out real regressions

When you add a static analysis tool to a CI pipeline, the first run often reports hundreds of existing issues. Those findings are useful for cleanup, but they also mask new problems that appear after a change. Teams end up scrolling through long lists or, worse, ignoring the output altogether.

Thesis: a Qodana baseline lets you suppress known issues while still detecting new ones

Qodana’s baseline feature stores the result of an initial analysis as a JSON file. On subsequent runs, Qodana compares the fresh report against that baseline and only shows issues that are new or have changed severity. This gives you a clean signal for regressions without removing the ability to run a full scan when needed.

Creating and committing a baseline

1. Make sure Docker is available and you have read/write access to the repository.

2. Run Qodana once to produce a full report and export it as a baseline:

   # From the root of your project
   # Adjust the image tag to match your Qodana version
    docker run --rm -v "$(pwd)":/data jetbrains/qodana-2024.3 \
      --baseline /data/qodana-baseline.json
   

   This command mounts the current directory into the container, runs the analysis, and writes qodana-baseline.json into the project folder.

3. Add the generated file to version control:

   git add qodana-baseline.json
    git commit -m "Add Qodana baseline for regression detection"
   

   Committing the baseline ensures that every CI agent and developer sees the same reference point.

Using the baseline in CI to see only new issues

With the baseline committed, a typical CI step looks like this:

# CI step (example for a generic script)
 docker run --rm -v "$(pwd)":/data jetbrains/qodana-2024.3 \
   --baseline /data/qodana-baseline.json

If the code introduces a fresh inspection violation, the output will list only that violation (or any severity changes). Previously known issues remain suppressed.

Worked example: verifying the baseline effect

1. Start from a clean state where the baseline exists and the CI passes (no new issues reported).

2. Introduce a small change that triggers a known inspection, for instance adding an unused import in a Java file:

   // src/example/Unused.java
   import java.util.List; // this line will be flagged as unused
   public class Unused { }
   

3. Run Qodana with the baseline flag:

   docker run --rm -v "$(pwd)":/data jetbrains/qodana-2024.3 \
      --baseline /data/qodana-baseline.json
   

   You should see a report that mentions the unused import and no other issues.

4. To confirm the baseline is working, run Qodana without the baseline:

   docker run --rm -v "$(pwd)":/data jetbrains/qodana-2024.3
   

   This full scan will list the unused import plus all the issues that were stored in the baseline, demonstrating that the baseline is suppressing the known set.

Trade‑offs and maintenance considerations

• Baseline size: The JSON file grows with the number of reported issues. In large codebases it can add several megabytes to the repository and slightly increase container startup time as Qodana loads the file.

• Staleness risk: If you intentionally fix or remove existing issues, the baseline will still contain entries for those problems until you regenerate it. New issues that appear in the same lines may be hidden until the baseline is updated.

• Update process: Treat baseline regeneration as a deliberate step. After a planned cleanup, run the baseline‑creation command again, commit the updated file, and push the change. Document this step in your contribution guide so teammates know when to refresh the baseline.

Practical way to check that your baseline is effective

1. Run the full scan (no --baseline) and save the output to full.txt.

2. Run the baseline scan and save the output to baseline.txt.

3. Use a diff tool to verify that baseline.txt is a subset of full.txt and that any differences correspond to the changes you made.

If the diff shows only the expected new violations, the baseline is working as intended.

Actionable closing

Add a baseline to your repository today:

1. Generate the initial qodana-baseline.json and commit it.
2. Update your CI configuration to always invoke Qodana with the --baseline flag.
3. Schedule a baseline refresh after each major cleanup sprint or when you upgrade Qodana versions.

With this practice in place, your CI will highlight regressions without being buried under historic debt, letting you act on real problems faster.