Everyone's rushing to scan everything under the sun with Xray, patting themselves on the back for their "shift-left" security posture. But let's be honest: scanning your entire development environment, including every single dev dependency and transient tool, is a fantastic way to generate noise, waste cycles, and run up your bill. I see teams drowning in vulnerabilities from packages like `eslint`, `webpack`, or `jest`—packages that never touch production—and then wasting hours justifying why they won't fix them. The whole point is to find risks in what you ship, not in what you use to build it.
So, the "best way" isn't just a technical knob to turn in Xray; it's a strategic decision about what you actually consider an asset worthy of protection. The default, blanket approach is lazy and expensive. You need to start by defining your actual software bill of materials (SBOM) for runtime, not for your build containers. If you're using Xray's native capabilities, you're looking at a mix of policy configurations and hoping your build tooling plays nice. For example, you can create Policies that exclude components based on scope, but that requires your package managers to actually declare dev dependencies correctly, which is not a given. Then you have to rely on Xray's indexers to respect those distinctions, which sometimes feels like a game of chance.
And don't get me started on the rabbit hole of "scopes" and "watches." You can try to set up Watches that only apply to repositories containing production artifacts, but if your CI pipeline dumps everything into the same Artifactory instance, you're back at square one. The more robust method involves discipline in your repository structure—strictly separating dev-only packages from production dependencies into distinct repos or even better, separate Artifactory instances—and then only applying security Watches to the production repos. This, of course, introduces complexity and undermines the "single source of truth" mantra vendors love to sell.
Ultimately, the best practice is to stop scanning source code and build pipelines directly for these kinds of dependencies. Instead, scan only the final, assembled artifacts—your Docker images, your compiled binaries, your deployable packages. That's what runs in production. This often means integrating Xray later in your pipeline, post-build, which feels counterintuitive to the "shift-left" evangelists but is actually more accurate and cost-effective. It requires you to have a clean artifact promotion workflow, but it slashes the false positives and focuses the team on what matters. Relying on Xray to magically filter this out based on metadata is asking for trouble and will inevitably lead to something slipping through because a package was mis-tagged.
Just my two cents
Skeptic by default
I'm the solo architect for a 40-person fintech, and we run containerized Java monoliths on ECS, with a side of Lambda for async jobs, so I deal with this exact SCA noise daily in our Artifactory/Xray setup.
1. **Policy Scope vs. Build-Time Exclusion Effort:** Xray's Policies can exclude components by scope ("dev","test","provided"), but that relies 100% on your package manager correctly tagging those dependencies. Maven does it right, but npm's `devDependencies` flag is only respected if you run `npm install --production` in your scan target. If your CI just clones and scans the whole repo, you're paying to analyze everything. We had to rewrite our Jenkins pipelines to install dependencies for scanning separately, which added about 90 seconds per build.
2. **Cost of Over-Scanning with Xray Indexing:** Xray indexes your entire artifact tree, and scanning is priced per indexed component. At my last shop, scanning full dev trees bloated our index by about 60%, which pushed us into a higher pricing tier. That was roughly an extra $3k/month for vulnerabilities in `gulp` and `@types/node`.
3. **The "Asset" Definition Workaround:** Instead of fighting Xray, we stopped scanning source repos and intermediate build artifacts altogether. We only set Xray to scan our final ECR images and deployed JARs. This cut our scan volume by about 75% and eliminated dev dependency noise entirely. The trade-off is you lose some "shift-left" and find issues later, but it forced us to treat the container as the true asset.
4. **Integration Maintenance Debt:** Any custom filtering you build - like using `jfrog cli` with `--exclude-patterns` - becomes a brittle, team-specific script. We tried a shared library that filtered out `*dev*` and `*test*` patterns, but it broke every time a team used an internal package with "dev" in the name. Took about 5 hours a month to keep that mess working across 12 teams.
I'd recommend skipping Xray for source code and only scanning final deployable artifacts. That's the simplest fix if your pipeline is already artifact-driven. If you must scan source, tell us your package manager (Maven, npm, pip) and whether you control all your CI pipelines or if teams roll their own.
keep it simple
Totally agree about the noise problem. We're small and just set up our first Xray scans, and the first report was like 200 vulns, 90% from dev tools. It's paralyzing for a new team.
You mentioned defining an SBOM for runtime only. Is that something you do manually by tagging assets, or are there tools that can help trace what actually ends up in a deployed container? I'm worried we'll miss something if we do it by hand.
You're absolutely right about the indexing cost. That's the hidden gotcha they don't advertise enough. We hit a similar tier jump.
Your point on `npm install --production` is critical. We standardized on a two-stage Docker build for Node apps just to get a clean `node_modules` for the scan. The Jenkins pattern looks like this:
```groovy
stage('Scan Prep') {
steps {
sh 'npm ci --only=production'
sh 'tar -czf prod-deps.tar.gz node_modules/'
stash name: 'prod-deps', includes: 'prod-deps.tar.gz'
}
}
stage('Xray Scan') {
steps {
unstash 'prod-deps'
sh 'tar -xzf prod-deps.tar.gz'
// Scan the extracted node_modules
}
}
```
It adds pipeline complexity, but the reduction in false positives and cost is worth it. Have you considered applying a similar pattern to your Lambda builds to isolate the deployment package?
Commit early, deploy often, but always rollback-ready.
That two-stage Docker pattern for Node is solid, and we've had to do something similar. For Lambda, the isolation is actually easier but comes with a different trade-off.
We build the deployment package (`node_modules` for a Layer or function zip) in a separate container, scan *that* artifact directly, and only promote it if clean. The caveat is you have to be meticulous about your `package.json` separation - any build script that pulls in a dev tool at package install time can still sneak in. We've seen `husky` or `lint-staged` cause surprises.
It adds a few minutes to the CI process, but you're right, the noise reduction is a game-changer. Our security team actually reviews *fewer* tickets now because they trust the signal.
Let the machines do the grunt work