Making Quality Gates Block PRs Without Holding Releases Hostage
Every service in the platform I work on runs SonarQube analysis in CI. For a long time the quality gate was purely informational: the scanner ran with sonar.qualitygate.wait=false, and a follow-up script fetched the gate status and printed a warning at most. A PR with a failing gate still showed a green build and could be merged. Everyone could see the debt growing; nobody was blocked by it.
I recently made the gate a blocking check on pull requests across all our shared pipeline templates. The interesting part is not the YAML, it is the policy design and one widely held misconception.
The misconception that blocks adoption
Whenever someone proposes making the quality gate mandatory, the same objection appears: “our codebase has years of legacy issues, the gate will fail everywhere and nobody can merge anything.”
That objection rests on a misconception. A SonarQube quality gate has two kinds of conditions: overall code conditions and new code conditions. When SonarQube analyzes a pull request, it evaluates only the new code conditions, and only against the lines the PR changed. The thousands of legacy issues on the main branch are not part of a PR’s verdict at all.
So a mandatory gate on PRs does not demand that anyone fix the past. It demands one thing: the code being added right now must not add new debt. No new blocker bugs, no new hardcoded credentials, a clean reliability rating on the diff. Touch nothing bad and you merge. That reframing turned the conversation from “this will block everyone” to “why was this ever optional.”
The new code conditions we settled on
Turning enforcement on was also the moment to audit the conditions themselves. This is the set we now run with, all on new code:
| Condition | Threshold |
|---|---|
| Coverage on new code | >= 80% |
| Duplicated lines on new code | <= 3% |
| Maintainability Rating | A |
| Security Rating | A |
| Reliability Rating | A |
| Security Hotspots Reviewed | 100% |
| Blocker Issues | 0 |
| Critical Issues | 0 |
Three changes came out of the audit, and each carries a lesson:
- Reliability Rating A was missing. Without it, a PR could introduce new bugs and still pass, because no other condition counts bugs. If your gate has Security and Maintainability ratings but not Reliability, new bugs sail through.
- Blocker Issues was tightened from “fails above 3” to zero. A gate that tolerates three new blockers per PR is not really a gate.
- A “Vulnerabilities > 3” condition was removed as redundant. Security Rating A already fails on the first new vulnerability, so the count condition could never fire first. Redundant conditions do not add safety; they add noise when people read the gate.
Blocked is not stuck
A blocking gate needs an escape hatch that is not “disable the check.” A developer whose PR is blocked has exactly two paths: fix the issue, or review it in SonarQube and mark it as False Positive or Accepted. Both outcomes are healthy. A fix removes the debt; a waiver records a conscious, attributable decision that a human looked at the finding and judged it acceptable. Either way the issue was handled, never silently merged past. Our pipeline’s failure message spells out both paths so nobody has to ask what to do next.
Two practical notes on the waiver path. First, restrict the “Administer Issues” permission to tech leads or senior developers, so a waiver is a reviewed decision and not a self-service bypass. Second, only issue-based conditions (issue counts and ratings) can be waived this way; coverage, duplication, and hotspot-review conditions cannot, and the only way past those is to actually write the tests or review the hotspots.
What you need for this to work
- Edition. PR analysis is a paid feature: SonarQube Developer Edition or higher. Community Edition analyzes branches only, so there is no PR verdict to enforce.
- No new code configuration for PRs. People assume they must configure a comparison branch. For pull request analysis they do not: SonarQube automatically defines new code as the diff against the PR’s target branch. The project-level New Code setting (previous version, days, reference branch) only affects branch analysis.
- What actually marks a run as PR analysis. Three scanner properties:
sonar.pullrequest.key,sonar.pullrequest.branch, andsonar.pullrequest.base. If the key is present, the server treats the run as a PR analysis. We never set these by hand; the SonarQube extension’s prepare task detects a PR build and injects them from the CI platform’s built-in PR variables. The same template therefore produces a PR analysis on PR builds and a plain branch analysis everywhere else. - Enough git history. A shallow checkout can break new code attribution, so the scanner must see the real diff.
Strict on PRs, lenient everywhere else
The enforcement is deliberately asymmetric:
- PR builds:
sonar.qualitygate.wait=true, so the analysis task itself fails on a red gate, plus a status-check step that exits non-zero as a backstop. Combined with a build validation policy, a gate-failing PR cannot be merged. - Main branch, release, and manual builds: unchanged, warn-only. A red gate must never block a production hotfix at two in the morning.
The switch is a compile-time expression on the build reason, so one template change applied the policy to a dozen pipelines at once.
Implementing it also surfaced a quiet pre-existing bug: the status-check script always queried the main branch’s gate status, even on PR builds, so PR runs were reporting the wrong verdict entirely. The fix was passing the pull request id to the SonarQube API so the check reads the analysis it actually belongs to.
Verify both paths before flipping the switch
Merging the template change was the enforcement switch-on for every team, so I verified it end to end first with a throwaway draft PR: a probe class containing a deliberate blocker bug and a fake hardcoded credential. Failure path confirmed (analysis failed, PR unmergeable), probe removed, pass path confirmed, test PR abandoned. Nothing ships a blocking check without seeing it block.
Takeaway
A quality gate that only warns is a dashboard. A quality gate that blocks PRs on new code conditions is a ratchet: the debt can shrink, but it can no longer grow. Understanding that PR analysis judges only new code, and that a blocked developer always has a fix-or-waiver path, is what makes “mandatory” a reasonable policy instead of a scary one.