Incremental Analysis vs Quality Gate Baseline: Enforcement Scope Mismatch in CI Pipelines

Root Cause

Qodana's incremental analysis mode evaluates only changed files, while quality gates compare new-issue counts against a project-wide baseline. When a modified library changes its API signature, downstream callers remain unevaluated in incremental runs, yet the gate sees no new issues in those untouched files. This creates an enforcement gap where cross-module regressions bypass detection.

Immediate Resolution

1. Run full-project analysis on merge events: Configure your CI pipeline to execute a complete Qodana scan when code merges to main or develop branches. Apply quality gates to this full run to catch cross-module impacts.
2. Retain incremental analysis for pull requests: Keep fast incremental feedback for PR authors, but do not use PR analysis results for gate decisions.
3. Implement absolute thresholds for small changesets: Replace percentage-only gates with absolute limits (e.g., zero new errors) to avoid unpredictable behavior in tiny PRs where a single issue can exceed or fall below percentage thresholds.
4. Establish baseline review cadence: Review and approve baseline updates on a regular schedule (weekly or per release). Never auto-update baselines on feature branches; require explicit pull requests for baseline diffs.

Verification Steps

Before implementing, confirm your analysis scope:

• Run qodana --version to identify the installed version.
• Check whether your project runs as a single Qodana analysis or separate per-module scans. Cross-module evaluation requires unified project scope.
• Create a test branch with an intentional API signature change in a library and an unchanged caller. Verify that a full analysis flags the caller while incremental analysis does not.

Configuration Pattern

Use conditional quality gate application based on branch context:

# Example GitHub Actions workflow snippet
name: Qodana Scan
on:
  pull_request:
    branches: [main]
  push:
    branches: [main, develop]

jobs:
  qodana-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: JetBrains/action-qodana-scan@v1
        with:
          incremental: ${{ github.event_name == 'pull_request' }}
          # Only apply quality gates on full scans
          fail-on-new-issues: ${{ github.event_name == 'push' }}
          # Absolute threshold for small changesets
          max-new-errors: 0

Missing Diagnostic

Confirm whether all modules are analyzed in one Qodana project run or as separate per-module runs. Separate runs cannot evaluate cross-module impacts even with full analysis. This determines whether quality gates can meaningfully catch architectural regressions.