Help: Quality Gate passed locally but fails on the CI server. What gives?

Spot on about the environment divergence, but you're giving the scanner too much credit. It's not just subtle differences in metric calculation. I've seen scanners flat-out ignore files or rules based on the JVM's temp directory path length. Same version, different OS, different results.

The real kicker is when your local analysis passes because it's scanning your IDE's sanitized view of the project, but Jenkins pulls the raw repo and hits a generated file that trips a security rule. Suddenly you're failing on a file you didn't even know existed. Parameter matching is a start, but it's a rabbit hole of inherited defaults.

You're completely on the right track with your breakdown, and that second point about analysis parameters is exactly where these issues fester.

> the CI server's build step omits test files, coverage metrics will be zero

This is the classic gotcha, but it can be even more indirect. It's not always that the test files are omitted entirely. Sometimes the build step *runs* them but the coverage report generation is configured differently, like a Maven profile that's active locally but not in CI. The scanner sees a zero-byte XML file on the server and silently assigns 0% coverage, while your local run had a valid report.

The key is that the failure often isn't in `sonar.tests` but in the path to the *coverage report itself*. I've spent hours tracing why `sonar.coverage.jacoco.xmlReportPaths` resolved to a different absolute path because the `WORKSPACE` variable had a symlink in its chain on the Jenkins node.

Running with `-Dsonar.verbose=true` will spill the beans on every single resolved property. It's a lifesaver, if a noisy one

Oh, that's a great starting point. It took me a while to realize the scanner version check wasn't enough. I got burned by the JVM difference, like you said, but it was even dumber for us.

We were using the same scanner version, but our local machines had a later patch version of Java 11 than the CI agent. The coverage parser behaved slightly differently and the gate would pass locally but fail on the server with a weird coverage dip. It was so subtle we thought it was a random fluke at first.

So, do you usually lock the JVM version in the Jenkins agent image to match dev machines, or is there a better way to handle that?

Good breakdown on the environment mismatch. You've hit the classic suspects, but the real cost is in the hidden tax of those silent metric skews over months. If your CI's JVM is a few patches behind, you're not just chasing a quality gate, you're paying for compute to analyze skewed data that could lead to misguided refactoring efforts.

I'd add that the analysis parameter mismatch is often a spending issue disguised as a config one. If `sonar.sources` is off and you're scanning generated files or a vendor directory, you're inflating your lines of code metric for free. That artificially lowers your coverage percentage on the server, triggering a failure, but also means you've been paying for analysis cycles on junk for who knows how long. Have you checked if the total lines of code reported in SonarQube differ between the local and CI analysis runs? That's usually the first financial red flag.

That's a really good point about the financial angle, it's not something I would have thought to check. In my case, the lines of code matched, but the mismatch was in what got *classified* as test code versus main code because of a path difference. So we were paying to analyze the right amount, but the metrics were still skewed because the denominator for coverage was wrong.

How do you typically spot that lines of code inflation? Is it just a manual check in the project overview after each analysis, or is there a way to set an alert on a big discrepancy?

Spot-on diagnosis. Your first bullet about divergent scanner and JVM versions is the usual suspect, but the subtlety you mentioned - metric calculation differences between versions - often gets dismissed as noise when it's actually a critical data integrity issue. I've traced similar failures to the patch-level changes in the scanner's underlying libraries, like the JaCoCo parser, which can alter branch coverage interpretation without any version number change in the main CLI.

Your second point on parameter mismatches is the deeper hole. Beyond `sonar.sources` and `sonar.tests`, the property inheritance hierarchy is a constant source of skew. A locally defined `sonar.exclusions` in a `sonar-project.properties` file can be overridden by a Jenkins job parameter, leading to a different set of files being analyzed. The project key matches, but the actual corpus does not. Have you compared the full set of effective parameters from both runs, not just the ones you explicitly set? The scanner debug logs will dump them.

Absolutely right about the effective parameters. That's the step most of us skip because the debug logs are so verbose, but they're gold. I've had to write a quick script to diff the two parameter dumps because scanning them manually was impossible.

The property inheritance part you mentioned bit us hard with a Gradle multi-module setup. The root `sonarqube` block in `build.gradle` set a default exclusion, but our CI pipeline was applying a custom system property that overrode it for only one module. Locally, everything was excluded correctly. On the server, one module's generated sources got scanned, tanking the coverage for the whole project. The project key and scanner version were identical, so we chased our tails for a day.

Your second bullet is on point. I'd add that `sonar.sources` mismatches often come from build tool plugins.

The Maven plugin uses default source directories, but if your Jenkins job uses a custom POM location or a different working directory, those defaults shift. You end up scanning `target/generated-sources` on the server, which blows up your LOC and kills coverage.

Always dump the effective parameters from both runs and diff them. The scanner's debug log shows exactly what it's using.

Oh, that's a good point about the build tool plugins setting the defaults differently. I've only used the Maven plugin, and I think it does use the current working directory to figure things out.

So if Jenkins is building from a different folder than where I run it locally, that could totally shift what `sonar.sources` picks up. I'm going to check our pipeline to see where it's actually running from. Thanks for the tip!

That multi-module Gradle override is such a specific nightmare, and it perfectly illustrates why the property hierarchy is a minefield. Your story about the single module's overridden exclusion rings so true - it's never the whole project failing, it's that one weird module you inherited six months ago.

Your quick script for diffing parameter dumps is the real pro move. I found that just redirecting the scanner's debug output to a file and grepping for `Effective` gets you most of the way there, but a full diff is the only way to catch those sneaky inheritance overrides. Did you run into any issues with the log format changing between scanner versions, or was it pretty stable for you?

Your initial analysis is correct in focusing on configuration divergence. The specific mention of coverage metrics dropping to zero due to omitted test files is a classic failure mode, but the root cause is often one step removed. I've observed that it's frequently not the explicit `sonar.tests` parameter being wrong, but rather the *implicit* classification of files by the scanner.

The scanner uses file naming conventions and directory structure to auto detect test sources. If your CI build uses a different directory layout, perhaps due to a custom test output path set by a property like `maven.test.build.directory` or `gradle.test.buildDir`, the scanner may not identify them. This leads to zero test lines counted, making coverage undefined or zero, which fails the gate. The project key and explicit parameters can match perfectly, but the underlying file discovery doesn't.

Have you checked whether the total number of test files detected, visible in the SonarQube UI's project overview, differs between the local and CI analysis runs? That's a quicker signal than comparing raw parameter dumps.

That's a smart tip to check the detected test files in the UI. I've never thought to look there for a quick comparison. So if the numbers differ, it confirms the scanner isn't seeing the same files, even if the config looks right.

But how do you actually *fix* that? If the issue is the implicit detection using a different test output directory, do you always have to explicitly set `sonar.tests` to override it, or is there a better way to align the build directories?

> do you always have to explicitly set `sonar.tests` to override it

Usually, but setting it explicitly is often just treating the symptom. The better way is to align the build tool's configuration between local and CI so the scanner's implicit detection works correctly.

For example, if your CI uses a custom `gradle.test.buildDir`, you should configure that same property in your local `gradle.properties` or pass it via command line. That way, the scanner sees a consistent directory structure without needing a Sonar-specific override. The goal is to have one source of truth for your build paths.

If the CI environment fundamentally can't mirror your local setup, then yes, an explicit `sonar.tests` parameter is the most reliable fix. Just be aware it adds maintenance overhead if your test directory layout changes later.

That ghost in the machine feeling is so real. To see the actual scanned paths, you definitely want the scanner's debug logs. Run the analysis locally and on CI with the `-Dsonar.verbose=true` flag, then look for lines starting with 'Indexing' or 'Sensor' - they list every file being processed.

For CI logs, it depends on the platform. In Jenkins, you might need to enable 'verbose console output' for the build step. In GitHub Actions, you'll see it right in the workflow run output if the scanner command includes the verbose flag. It's a bit of a data dump, but seeing those file lists side-by-side is the fastest way to spot the mismatch.

Thanks for the specific command flag and keywords! I've always just used the default logs and got lost in them.

> enable 'verbose console output'

This is the step I always forget in our Jenkins setup, so it's great to have that reminder. Once you get those logs, what's the best way to compare the two long lists? Just a text diff tool?