Question
Incremental Analysis vs Quality Gate Baseline: Enforcement Scope Mismatch in CI Pipelines
Rin Dey
0 reputation · 18 Jul 2023, 03:40 UTC
115.5K views
Loading…
0 reputation · 18 Jul 2023, 03:40 UTC
Qodana's incremental analysis mode (default since 2023.2) re-analyzes only changed files to accelerate CI feedback, while quality gate evaluation continues to reference the whole-project baseline for threshold decisions. This creates an enforcement gap: cross-file dependency issues — such as API signature changes affecting downstream callers — may go undetected during incremental runs, yet the quality gate still passes because new-issue counts remain low.
Percentage-based fail thresholds on new code behave unpredictably in small PRs where a single violation exceeds the allowed percentage. Baseline auto-update in "new issues only" mode can silently accept regressions if not reviewed periodically. The documentation does not formally specify how incremental analysis interacts with quality gate evaluation across module boundaries, and behavior may change between releases.
Establish a CI configuration that preserves fast incremental feedback while ensuring quality gates catch cross-file regressions and architectural debt accumulation without generating notification noise from legacy violations.
What configuration pattern ensures quality gates evaluate cross-module impacts during incremental runs? How should percentage-based thresholds be calibrated for small changesets to avoid false passes or excessive failures? What baseline review cadence mitigates silent regression acceptance in "new issues only" mode?
A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 18 Jul 2023, 08:11 UTC
Run incremental analysis on every PR to keep feedback fast. After the PR passes, trigger a full analysis on the target branch (e.g., main) and compare the SARIF outputs. Gate on both the new‑issue count from the incremental run and the baseline delta from the full run. This dual check surfaces cross‑module regressions that incremental mode would otherwise miss.
Treat any baseline XML change as a review item. Merge the baseline only after a code‑review approval, and run a periodic full analysis (weekly or before major releases) to surface silently accepted regressions.
qodana