Help: Quality Gate passed locally but fails on the CI server. What gives?

Harper Adams · 2026-08-07T17:01:23Z

I've encountered a persistent and subtle issue in our current pipeline setup that I believe warrants a thorough discussion. Despite a local analysis with the SonarQube Scanner showing all Quality Gate conditions as passed, the same project fails its Quality Gate when analyzed on our CI server (Jenkins, in this case). The code, branch, and SonarQube project key are identical. This discrepancy points to environmental or configuration divergence, not a tool bug per se. After methodically comparing the environments, I've isolated several potential root causes. The most common, in my experience, are: * **Divergent SonarQube Scanner versions or runtime environments:** The local machine might be using a different version of the scanner (e.g., SonarScanner CLI) or a different Java Runtime Environment than the CI agent. Subtle differences in how metrics are calculated can emerge between versions. * **Analysis parameter mismatches:** While the project key is the same, other parameters passed to the scanner can differ. Crucially, the `sonar.sources` and `sonar.tests` definitions must be consistent. If the CI server's build step omits test files, coverage metrics will be zero, causing a failure. * **Historical data and branch handling:** This is particularly relevant if you are using branch analysis or PR decoration. The CI server might be analyzing the branch with a different `sonar.branch.name` property, causing it to compare against a different baseline than your local analysis. Furthermore, if the CI job cleans the workspace before analysis, it may not have access to the same execution data (e.g., JaCoCo reports) generated earlier in the pipeline. To facilitate debugging, I recommend executing the following diagnostic steps on both systems and comparing the outputs. First, ensure you are capturing the full debug output from the scanner. For Maven: ```bash mvn sonar:sonar -Dsonar.verbose=true -X > local_analysis.log ``` For SonarScanner CLI: ```bash sonar-scanner -Dsonar.verbose=true -X > local_analysis.log ``` Then, meticulously compare the following sections from the local and CI logs: 1. The "Analysis properties" section at the start of the report. 2. The computed metrics for the key failing condition (e.g., "Coverage on New Code is 0.0%"). 3. The paths resolved for `sonar.sources`, `sonar.tests`, and any report paths (like `sonar.jacoco.reportPaths`). In our specific case, the culprit was the CI server's build configuration using a different `sonar.java.binaries` property, which led to the analysis skipping pre-compiled bytecode and thus failing the "No Duplicated Blocks" condition. The local analysis, using the default Maven lifecycle, had the correct class directory. Has anyone else conducted a similar forensic comparison and identified other less-obvious parameters—perhaps related to memory settings, timeouts, or the handling of `.sonarlint/` settings—that can create this illusion of passing gates locally but failing in CI? I am particularly interested in edge cases within containerized or ephemeral CI environments.

Page 2 / 2 Prev

SonarQube Reviews

Last Post by claraj 52 minutes ago

16 Posts

15 Users

0 Reactions

1 Views

RSS

claraj

(@claraj)

Estimable Member

Joined: 3 weeks ago

Posts: 172

08/08/2026 11:17 am

Spot on about the environment divergence, but you're giving the scanner too much credit. It's not just subtle differences in metric calculation. I've seen scanners flat-out ignore files or rules based on the JVM's temp directory path length. Same version, different OS, different results.

The real kicker is when your local analysis passes because it's scanning your IDE's sanitized view of the project, but Jenkins pulls the raw repo and hits a generated file that trips a security rule. Suddenly you're failing on a file you didn't even know existed. Parameter matching is a start, but it's a rabbit hole of inherited defaults.

Prove it

ReplyQuote

Page 2 / 2 Prev

Forum Icons: Forum contains no unread posts Forum contains unread posts

Topic Icons: Not Replied Replied Active Hot Sticky Unapproved Solved Private Closed