We recently conducted a comparative analysis of FOSSA and Synopsys Black Duck, focusing specifically on their false positive rates for license and policy violations. The test suite consisted of 200 curated open-source packages with known, unambiguous licensing metadata, designed to stress-test dependency discovery and license attribution logic.
Our methodology involved scanning the same codebase snapshot with both tools using their default policies, then manually verifying each flagged issue. The environment was a moderately complex Node.js and Go monorepo with ~500 direct dependencies.
The raw results were as follows:
```
Tool Total Flags False Positives FP Rate
FOSSA 47 5 10.6%
Black Duck 68 19 27.9%
```
The majority of FOSSA's false positives stemmed from misinterpreting a "BSD-2-Clause" license as "BSD-3-Clause" in two instances, and flagging three dual-licensed packages (MIT/Apache-2.0) as potential conflicts when our policy allowed either. Black Duck's false positives were more varied, including several cases where it flagged common permissive licenses (e.g., MIT) as "unreviewed" and generated alerts for packages that were development dependencies but excluded from our build artifacts.
A key differentiator appeared to be in the dependency resolution phase. FOSSA's deeper tracing into vendored and transitive dependencies produced more accurate graphs, reducing phantom violations from missed indirect packages. Black Duck's broader, pattern-matching approach seemed to cast a wider net, catching edge cases but also generating more noise.
For teams operating in regulated environments or with strict compliance requirements, a lower false positive rate directly translates to reduced manual triage overhead. While both tools require policy tuning, our findings suggest FOSSA's default heuristics may align more closely with canonical license data, providing a more efficient starting point.
—J
—J
Interesting results, especially that difference in FP rates. I've seen FOSSA be a bit more conservative with its default policies, which might explain the lower count.
Black Duck flagging MIT as "unreviewed" on default settings is a bit surprising though. That's usually a green light everywhere. Did you find any pattern in the other 16 false positives, like were they mostly in the Go modules or scattered? Node's package.json tends to be cleaner for license detection in my experience.
The dual-license flagging from FOSSA is a known quirk - you often have to tweak the policy to accept any from a set, not just one. Once you adjust that, those false positives typically drop off.
ABT – always be testing
The difference in how they treat MIT flags is really telling. We've had similar Black Duck false positives on default scans where it holds back common licenses for "review" by default, even obvious permissive ones. FOSSA's more aggressive default approval list definitely helps its FP rate.
That said, the BSD-2/BSD-3 mix-up is a weird one for FOSSA - I haven't seen that before. Was the license file ambiguous, or was it a clear SPDX identifier? Could be a parser bug in a specific version.
Once you tune the dual-license policy in FOSSA, those false positives vanish, as you noted. The real question is how much time you want to spend tuning defaults versus accepting a higher initial FP rate.
spreadsheet ninja
So these false positives you're seeing, are they mostly from the Go side? I'm trying to learn and I've heard Go's license detection can be trickier because of the go.mod file.