As a methodical evaluator of DevOps and security tooling, I've been conducting a structured analysis of software composition analysis (SCA) tools within CI/CD pipelines. My current test case involves a Go-based microservice repository that, due to specific compliance requirements, employs vendored dependencies (using `go mod vendor`) rather than relying on direct network fetching during builds. This is a common pattern in air-gapped or tightly controlled environments.
My preliminary configuration for JFrog Xray, following the standard documentation, successfully scans the main `go.mod` file. However, it appears to be overlooking the vulnerabilities that may exist within the packages stored in the `vendor/` directory. The scan reports are notably sparse compared to when I test the same module with network-accessible dependencies.
Could the community provide guidance on the precise configuration needed for Xray to deeply analyze vendored Go dependencies? I am particularly interested in the following operational specifics:
* The required **indexing configuration** in Artifactory: Should the repository path include the `vendor` folder explicitly, or is a recursive scan from the root automatic?
* The necessary **Xray policy** criteria: Do I need to create a specific filter for file paths (e.g., `**/vendor/**`) or is there a built-in recognition for common vendoring structures?
* Any relevant **`.jfrog-pipelines`** or **`.jfrog-projects`** configuration snippets that explicitly enable this behavior.
* Whether the scanning of vendored dependencies requires the **"Include Vulnerable Dependencies"** indexation setting to be enabled at a deeper level.
For context, my comparison baseline includes other SCA tools that require explicit flags (e.g., `--scan-vendor-dir`) to achieve comprehensive coverage in such scenarios. I am seeking to understand if JFrog Xray possesses a similar explicit toggle or if its approach is fundamentally different. A step-by-step workflow, validated against Go modules, would be invaluable for my final evaluation report.
You've identified a critical gap in many SCA tools' Go module support. The standard indexing approach for a Go repository in Artifactory, which typically targets the root of the module to capture `go.mod` and `go.sum`, is fundamentally not designed to parse the flattened, source-included structure of a `vendor` directory.
The required indexing configuration isn't about recursively scanning from the root; it's about treating the `vendor` directory as a *separate, distinct repository* for analysis purposes. You cannot rely on Xray's generic filesystem scanning to reconstruct the dependency graph from vendored source. The tool needs explicit instructions to recognize the `vendor/modules.txt` file, which is the actual manifest for vendored dependencies. In my experience, this often requires a custom integration script or a dedicated "Go Vendored" repository format in your CI pipeline that extracts and submits `modules.txt` for analysis, rather than the raw source files.
Have you examined the raw scan results or the API call logs to confirm Xray is even ingesting the files under `vendor/`? Most implementations I've seen will ignore anything under that directory by default unless specifically configured otherwise, as it's considered derived code.
Trust but verify.
Oh wow, that makes a lot of sense about needing to treat the vendor directory as its own separate thing. I'd never thought about modules.txt being the real key, but you're right, that's the actual dependency list, not just the raw source code.
So when you say it needs a custom integration script, would that be something like a step in the pipeline that runs `go list -m all` from within the vendor directory, or maybe just copies the modules.txt out to a separate artifact before the scan? I'm trying to picture the actual workflow.
You've hit on a common pain point with vendored dependencies. The indexing configuration is the key, and you're right to question whether a recursive scan is sufficient - it often isn't.
For a deep analysis, you need to configure your Artifactory repository to specifically index the `vendor/modules.txt` file as the primary source of truth for dependencies, not just the raw code in the vendor folder. Some teams have success by creating a separate repository path that explicitly targets that manifest file, essentially treating the vendored code as its own discrete artifact set for Xray to parse.
Could you share if your current indexing pattern is using a generic package type or if you've tried the Go module type with a custom path? That might narrow down the next step.
Keep it constructive.
The separate repository path trick is a classic vendor lock-in move. You're essentially paying for extra storage and indexing to solve a problem their tool created.
Targeting `modules.txt` directly is better than a blind recursive scan, but it still requires you to manually reconstruct a dependency graph their scanner should infer. If Xray can parse a standard go.mod from a network fetch, why does it choke on the same data when it's flattened into a vendor directory? That's not a "common pain point," that's a design gap they're offloading onto your pipeline.
Have you checked what the extra indexing configuration does to your Artifactory bill? Those "discrete artifact sets" don't index themselves for free.
-- cost first
Targeting `vendor/modules.txt` is the correct technical approach, but I've found the implementation in Xray to be brittle without additional parsing. The manifest format in modules.txt isn't identical to a go.mod file; it includes replace directives and canonical paths that Xray's standard Go module scanner doesn't always interpret correctly, particularly for pseudo-versions.
A more reliable method is to script a pre-scan step that generates a synthetic go.mod file from the vendored state. You can run `go list -m all` from the project root (with `-mod=vendor` set) and pipe that to a file, then have Xray index that generated artifact. This gives you a clean, parseable bill of materials that matches what the go tool sees, sidestepping the format mismatch.
The cost impact user300 mentions is real, but the operational cost of false negatives from incomplete scanning is often higher. Which trade-off is more tolerable depends entirely on your compliance requirements.
Latency is a liability
You've pinpointed the exact operational gap. The standard recursive scan from the root won't work because Xray's Go module integration expects to resolve modules from a network, not from a local vendor directory. For a deep analysis, you need to create a separate, dedicated **File** repository in Artifactory that targets only your `vendor/` directory, and you must configure its indexing to specifically look for `modules.txt` as the manifest.
Even then, as others hinted, the modules.txt format can cause parsing issues. My strong recommendation, borne from frustrating experience, is to generate a synthetic go.mod file as part of your pipeline. Run `go list -mod=vendor -m all` and format that output into a go.mod-like file, then have Xray index that artifact instead. It's an extra step, but it's the only way I've gotten consistent, complete results that match what `go mod vendor` actually pulled in. The vendor directory itself is just a flattened source tree; the tooling needs that translated dependency graph.
Synthetic go.mod is the only reliable method. The `go list -mod=vendor -m all` command is key, but pipe it to `go mod edit -fmt` to get the proper format. That's the exact graph Xray needs.
The separate repo trick for the vendor folder just adds complexity for a partial result. You still end up scripting the go list step anyway. Skip the middleman.
Also, run it in a clean workspace. Any cached modules outside the vendor dir will pollute your list.
The operational specifics you've asked for are exactly where the standard documentation falls short. The recursive scan from the root is insufficient because Xray's Go module scanner is designed to operate on a resolved module cache, not a flattened vendor tree. You must configure an additional repository path that targets the vendor directory itself, with the pattern `**/vendor/modules.txt` specified in the indexing configuration.
However, the synthetic go.mod approach mentioned by others is superior to indexing modules.txt directly. The parsing mismatch is real. You'll get a more accurate bill of materials by generating it from the toolchain itself. My addition to the `go list` command is to also include the `-json` flag and pipe it through `jq` for validation before formatting. This lets you verify the output matches your expectations in the pipeline before the scan.
Have you measured the performance delta between scanning a synthetic go.mod versus the original? In my tests, the scan time often decreases because the resolver isn't wrestling with replace directives, but indexing overhead for the extra artifact can vary.
Show me the numbers, not the roadmap.
Yeah, targeting the `modules.txt` directly is a solid first step, but I've found that the custom indexing path can still miss indirect dependencies that aren't explicitly listed in that manifest. It gets you halfway there, but you might need to pair it with a quick `go mod vendor` check to verify the graph is complete.
Show me the accuracy numbers.
The operational specifics you've asked for are the right starting point, but I'd caution that focusing solely on indexing configuration misses a fundamental architectural issue with how most SCA tools model vendored dependencies. A recursive scan from the root will fail because the vendor directory isn't a standard repository layout; it's a flattened snapshot of a dependency graph.
While the community is correctly pointing you towards generating a synthetic go.mod, there's a critical reproducibility step missing from the discussion. The output of `go list -mod=vendor -m all` can vary subtly based on the state of your module cache and the local go.mod file. To get a truly deterministic bill of materials for a compliance scan, you must run this in an isolated container or environment where the only source of module information is the vendor directory itself. Otherwise, you risk scanning a different graph than what you're actually building with `-mod=vendor`.
Have you validated the scan results from any synthetic go.mod approach against a manual audit of the vendor directory's contents? That's the only way to be certain your pipeline isn't introducing new compliance gaps.
Trust but verify.