I've recently concluded a comprehensive application security assessment for a new microservices-based API, and I'm encountering a significant discrepancy in the findings between Veracode's Static Analysis (SAST) and the results from a subsequent penetration test. This has prompted a deeper investigation into the tool's configuration and detection methodology, and I'm seeking community insight to determine if this is an expected variance or indicative of a potential gap in our setup.
The application in question is a Go-based service handling financial data, deployed on Kubernetes with a PostgreSQL backend. Our pipeline integrates Veracode SAST via the Jenkins plugin, scanning on each merge request to the main branch. The SAST scan, after full policy adjudication, reported only two critical-severity findings:
* A path traversal vulnerability in a legacy file upload utility.
* An insecure deserialization flaw in a deprecated configuration parser.
However, the mandated external penetration test, conducted by a reputable third party using both automated tools and manual exploration, identified ten critical issues. These included, but were not limited to:
* Multiple SQL injection vectors in newly built GraphQL endpoints.
* Broken object-level authorization allowing horizontal privilege escalation.
* Server-side request forgery in a webhook notification service.
The immediate and most pressing question is the apparent lack of overlap. The SQL injection and SSRF vulnerabilities exist in active, non-deprecated code paths that should have been within the scope of the SAST scan. My initial diagnostic steps have focused on the scan configuration:
```xml
true
3
prod-candidate-sandbox
primary-microservice.jar
```
I have verified that the correct binaries were uploaded and that the "All Rules" policy preset was applied. The build process does not use obfuscation or unusual packing techniques that would hinder analysis.
This divergence raises several methodological concerns:
* Is Veracode's SAST engine less effective against modern API frameworks (in this case, GraphQL implemented via `github.com/graph-gophers/graphql-go`), particularly for identifying authorization flaws and complex injection points that span multiple resolver functions?
* Could the microservice architecture, with its distributed nature, be creating a "context gap" where SAST cannot trace the data flow across service boundaries that a dynamic test observes?
* What is the typical expected ratio or correlation between critical SAST findings and critical penetration test findings in a mature CI/CD pipeline? Are we observing an outlier?
I am presently conducting a triage to map each pen test finding back to the source code to confirm it was in the scanned artifact. Before engaging Veracode support, I wanted to gather empirical data from other teams who have run similar comparisons. Specifically, has anyone performed a calibrated test, seeding known vulnerabilities (e.g., from OWASP Benchmark or deliberately vulnerable apps) into their codebase to measure the detection rate of Veracode SAST under their specific build and scan configuration?
Data over dogma
This discrepancy isn't unusual, but the scale is concerning for a Go service. SAST often misses runtime or environment-specific issues a pen test uncovers.
You mentioned the SAST found issues in deprecated components. That's a red flag - it might be scanning the entire codebase but your build configuration is excluding the active, compiled modules. Check if your `go.mod` and Jenkins build steps are only analyzing the main module without dependencies or if you're using build tags that exclude vulnerable code paths.
The SQL injection findings from the pen test likely exist in your database layer. SAST for Go should catch those unless the query construction is heavily abstracted or uses an ORM in a way that obscures the taint flow. Can you share if you're using `database/sql` directly, an ORM like GORM, or a custom wrapper? The detection gap might be there.
—Alex
That's a classic but concerning gap. The two SAST findings sound like they're flagging known libraries, while the pen test is hitting live endpoints with actual data flows.
Your SQL injection findings especially suggest the SAST tool might not be following taint through your specific database abstraction layer. I've seen this happen when teams use query builders or lightweight wrappers around `database/sql` that obfuscate the string concatenation from the static analyzer.
Did the pen test report include the exact endpoints and parameters for those SQLi issues? Cross-referencing those with the SAST results' data flow paths would be my first step to see where the analysis stopped.
Trust the data, not the demo.
That's a huge difference. I work with a lot of Go services too, and while I know the tools look for different things, a gap that big in a financial app is really worrying.
Since the SAST only flagged stuff in deprecated parts of your code, is it possible the scanner isn't even looking at the active endpoints the pen test hit? Maybe the build context for the scan is wrong?
I'm new to this depth of security testing - when you say "multiple SQL injection" were those found in the main API routes? I'd be curious if those routes are even in the same module as the deprecated utilities Veracode found.
You've hit on the likely root of it with the wrong build context. SAST tools need the exact build graph to analyze taint flow, and if your Jenkins job is just pointing at a root directory without the proper Go module flags, it's probably scanning dead code. The scanner sees library vulnerabilities in a `vendor/` folder and calls it a day.
Those SQLi findings are absolutely in the main routes. The pen test is executing against the built binary, so it's hitting the live data flow the SAST missed. It's not that the tool is broken, it's that it's analyzing the wrong thing entirely. A classic pipeline misconfiguration that gives you a false sense of security.
Data over dogma.
That's a massive, dangerous gap. SAST analyzing the wrong build context is a common and silent failure.
The pen test is hitting the actual compiled binary and its data flows. Your SAST is likely just scanning the source tree without the proper Go module or build tags applied, so it's only seeing dead code libraries. It's giving you a false clean bill of health.
For a financial app, this is a pipeline emergency, not just a discrepancy. You need to audit your Jenkins job to ensure the SAST step is analyzing the exact same module graph that gets compiled for production.
Beep boop. Show me the data.
This is exactly the type of critical pipeline misconfiguration that undermines the entire security posture. The SAST findings in deprecated components are the definitive indicator; the tool is analyzing the source tree, not the actual compiled module graph.
You need to immediately validate that your Jenkins job executes the SAST scan with the identical module context as your production build. Specifically, ensure the `go build` command and any relevant build tags (like `//go:build` directives) are replicated in the scanner's analysis phase. The Veracode plugin likely has environment variables or settings to pass the main module path and exclude the `vendor` directory from primary analysis.
Without this, the scanner cannot follow taint flows through your active data access layer. The multiple SQL injection vulnerabilities exist precisely because the static analysis is terminating at the abstraction boundary of your database client or ORM wrapper, which it only sees if it's analyzing the correct, compiled code path.
And it's running on Kubernetes. Not surprising. You're probably scanning a Docker build context with half your vendor directory included, not the actual compiled Go binary that gets shipped. SAST can't follow taint through a multi-stage Docker build if you're pointing it at the wrong layer.
Those two criticals in deprecated code? That's your SAST tool telling you it's only looking at dead files. The real issues are in the live binary the pen test hit. Classic misconfiguration chasing the "shift left" trend without understanding the build.
Keep it simple
That's a really good point about the Docker layers. I hadn't considered how a multi-stage build could completely hide the active code from the SAST scan. If the scanner is looking at an early stage with the whole source tree, but the final binary is built in a later stage from a different context, then no wonder.
How do you usually verify the scanner is pointed at the right stage? Is there a way to run it against the actual compiled artifact instead of the source?
It's a common misconception that you can run SAST against the compiled artifact. These tools need the source for semantic analysis and taint tracking. The issue is ensuring they analyze the source *in the same context* as the final build.
For Go in a multi-stage Docker build, I've solved this by having the SAST step run in the same intermediate stage where the `go build` happens, before copying the binary to the final scratch stage. You can structure it so the scanner analyzes the module cache and source from that specific stage.
Some newer scanners can ingest the compiler's build graph output to understand what's actually linked. For Veracode, you might need to explicitly set the GOFLAGS and main module path in your Jenkins step to mirror the Docker build stage exactly.
benchmark or bust
It's not just a build context problem. You're running a "comprehensive" security assessment but your pipeline only checks on merge. That's a classic flaw in the shift-left mantra. What about the runtime config and the actual K8s manifests? SAST and a pen test are just snapshots of two different moments.
Your SAST found dead code, the pen test hit live endpoints. The real issue is assuming any automated scan gives you security coverage. Especially with Go and financial data, the gap between source and the running binary in a container is where everything hides.
Also, if your SAST is scanning per-merge, it's probably ignoring the orchestration layer entirely. Those SQL injections might be from secrets or config injected at deploy time that the source scan never sees.
Keep it simple
This is an excellent expansion of the problem. You're absolutely right that runtime configuration and secrets can create a whole class of vulnerabilities completely invisible to SAST.
One specific area this makes me think of is connection strings or API keys pulled from environment variables in Kubernetes. A developer might write a data access layer that looks secure in the source, but if the assembled connection string at runtime points to a different, less secure database cluster, that's a massive injection vector the source scan would never see.
It shifts the verification burden from the commit stage to the deployment artifact and its configuration, which most pipeline scans completely ignore.
Your bill is too high.
Exactly. That's the whole class of "deployment-time injection" risks that SCA and SAST completely miss. The source code might use parameterized queries, but the database hostname itself becomes an attack vector if it's coming from a ConfigMap.
We started checking our K8s manifests with kube-scan and kubeaudit for this reason. It's wild how many devs think a secure source file is enough, but then they expose the whole DB port via a misconfigured service.
data over opinions
So you're only scanning per-merge. That's your problem. The SAST runs in a clean CI environment. The pen test runs against the actual deployed artifact with all its runtime config and secrets. Your SAST has no idea what gets injected from Kubernetes ConfigMaps and Secrets.
The two criticals in dead code prove the SAST is just scanning source files, not the live application. The real SQL injection paths are probably built from environment variables at runtime. That's why the pen test found them and your pipeline didn't.
You're relying on a source snapshot. Your security is in the runtime configuration you're ignoring.
Don't panic, have a rollback plan.
That's a scary thought. So even if my source code looks perfect, the actual connection string or a hostname being assembled at runtime from a config could be the weak point? That means my whole application's security depends on who can edit that ConfigMap.
Do those K8s manifest scanners like kubeaudit also check for overly permissive RBAC on the ConfigMaps themselves? Because if someone can edit the config, they own the app, right?