Configuring Codacy Quality Gates to Block Risky Pull Requests

The Problem: Merging Code That Degrades Quality

Teams adopt static analysis tools but still merge pull requests that introduce new lint violations, drop test coverage, or add security findings. The missing piece is an automated gate that evaluates only the changes in the PR and blocks merge when thresholds are crossed. Codacy provides this through codacy.yaml quality gates that integrate with GitHub, GitLab, or Bitbucket branch protection.

How Codacy Quality Gates Work

On each pull request, Codacy clones the repository, runs its analysis engine against the diff, and compares results against gates defined in codacy.yaml at the repository root. The gate evaluation produces a status check (e.g., Codacy/PR Quality Gate) that appears in the PR checks list. When you add that check to branch protection rules, a failing gate prevents merge.

Key concepts:

• Patterns — enable or disable specific lint rules per language (e.g., ESLint no-unused-vars, Pylint C0114).
• Quality gates — named sets of conditions with metrics, operators, and thresholds.
• New-code scope — gates evaluate only lines added or modified in the PR, not the entire codebase.

Worked Configuration: codacy.yaml

Place this file at the repository root. The example below enforces zero new issues and allows no more than a 0.5% coverage drop per PR.

version: '2'
patterns:
  JavaScript:
    eslint:
      no-unused-vars: true
      eqeqeq: "error"
  Python:
    pylint:
      C0114: false  # disable missing-module-docstring
quality_gates:
  - name: "PR Quality Gate"
    conditions:
      - metric: "new_issues"
        operator: "<="
        value: 0
      - metric: "coverage_change"
        operator: ">="
        value: -0.5

Explanation:

• version: '2' — current schema version as of 2024; validate locally with codacy-cli validate before committing.
• patterns.JavaScript.eslint — keys must match Codacy's internal language identifiers exactly (JavaScript, not JS). Rule names are the upstream tool's identifiers.
• quality_gates[0].conditions — each condition references a metric Codacy computes for the PR diff. new_issues counts issues introduced in changed lines; coverage_change measures percentage-point difference between base and head coverage.
• Operators: <=, >=, <, >, ==, !=. Values are numbers; for coverage change, negative means a drop.

Required CI Setup for Coverage Gates

The coverage_change metric requires a coverage report uploaded during CI. Codacy supports LCOV, Cobertura, JaCoCo, and others. Example GitHub Actions step:

- name: Upload coverage to Codacy
  uses: codacy/codacy-coverage-reporter-action@v1
  with:
    project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
    coverage-reports: coverage/lcov.info
    # optional: force-coverage-parser: lcov

Run this after your test suite generates the report. Without an uploaded report, coverage_change stays null and the gate condition passes unexpectedly — a silent failure mode.

Branch Protection: The Step Everyone Forgets

The gate runs and posts a status check regardless, but it only blocks merge when added to branch protection.

1. GitHub: Settings → Branches → Branch protection rules → Add rule → Require status checks to pass → Search for Codacy/PR Quality Gate (exact name from your gate's name field).
2. GitLab: Settings → Repository → Protected branches → Add protected branch → Allowed to merge: Codacy/PR Quality Gate status check.
3. Bitbucket: Repository settings → Branch restrictions → Add restriction → Require checks to pass → Select Codacy check.

Without this, developers see a red check but can still merge.

Limits You Should Know

• New-code only. Pre-existing violations in untouched files never trigger a gate. This is by design but surprises teams expecting whole-repo enforcement.
• Maximum 20 gates per repository. Plan gate granularity accordingly.
• No custom metrics or cross-repo policies. Conditions reference only Codacy's built-in metrics (new_issues, coverage_change, duplication_change, complexity_change, etc.).
• Monorepo constraint. A single codacy.yaml at the root applies to all languages. Per-subproject gates are not supported; consider separate Codacy projects per service.
• Free tier quotas. 1 private repository, 30 analyses/month. Gates stop running once quota is exhausted.

Common Mistakes and How to Avoid Them

1. Language identifier mismatch

Using JS, TS, or Py instead of JavaScript, TypeScript, Python causes rules to be silently ignored. Verify identifiers in Codacy's documentation or via codacy-cli analyse --directory . --format sarif locally — the output shows which language detectors fired.

2. Expecting gates to catch existing debt

A PR that touches a file with 50 pre-existing pylint violations but adds zero new ones passes the new_issues <= 0 gate. If you need whole-file enforcement, run the linter directly in CI as a separate step.

3. Missing coverage upload

As noted above, coverage_change becomes null without an uploaded report. Add a CI check that fails if the coverage artifact is missing, or use Codacy's API to verify upload succeeded.

4. Token rotation breaking gates silently

The CODACY_PROJECT_TOKEN must stay current. Rotate it in Codacy UI → Project → Settings → Integrations, then update the secret in your CI provider. No error appears in the PR; analyses simply stop running.

5. Upstream rule identifier changes

ESLint, Pylint, and other tools rename or remove rules across versions. Pin tool versions in your CI (e.g., eslint@8.57.0) and test gate behavior after upgrades.

Verifying the Gate Works

1. Create a test repository and add the codacy.yaml above.
2. Push a branch with a deliberate violation — e.g., a JavaScript file containing const unused = 1;.
3. Open a PR. In the Checks tab, wait for Codacy/PR Quality Gate to complete. It should fail with new_issues: 1.
4. Confirm merge is blocked by branch protection (the merge button stays disabled).
5. Fix the violation, push again, and verify the gate passes and merge becomes allowed.

For local preview before pushing, run:

npx codacy-cli analyse --directory . --format sarif --output codacy-results.sarif

This requires Node.js and the codacy-cli package. It runs the same analyzers Codacy uses and outputs SARIF you can inspect. Note: local analysis does not evaluate quality gates; it only shows detected issues.

Checking Gate History and Metric Values

In Codacy UI, navigate to Repositories → [your repo] → Quality Gates. Each PR evaluation shows:

• Gate name and pass/fail status
• Metric values at evaluation time (e.g., new_issues: 3, coverage_change: -1.2)
• Which conditions passed or failed

Use this to debug unexpected passes (often coverage_change: null) or to tune thresholds.

Schema Changes and Maintenance

Codacy evolves the codacy.yaml schema. The version: '2' field is current as of 2024. Before upgrading your CI pipeline or after a Codacy platform update:

• Run codacy-cli validate against your file.
• Review the Codacy changelog at blog.codacy.com/changelog for breaking changes.
• Test in a non-production repository first.

Summary Checklist

• codacy.yaml at repo root with correct language identifiers
• Quality gate conditions matching your risk tolerance
• Coverage reporter in CI uploading supported format
• Codacy status check added to branch protection rules
• CODACY_PROJECT_TOKEN stored as secret and rotated with process
• Local validation via codacy-cli validate in pre-commit or CI

With these in place, every PR that introduces new lint issues or drops coverage beyond your threshold will be blocked at the merge button — no manual review required.