Alright, let's cut through the vendor-generated blog posts and "revolutionary AI" press releases. Everyone's pushing their automated code reviewer as the next big thing, but when you're dealing with C++—with its templates, move semantics, undefined behavior landmines, and legacy codebases—the false positive rate can quickly turn your PR queue into a digital landfill.
I've been running a few of these tools (Sonar, DeepSource, Snyk Code, and that new GitHub-native one) in a shadow mode on our monorepo for the past quarter. We're talking a few million lines of modern and not-so-modern C++ (yes, there's still some C++11 lurking). The marketing claims about "context-aware" analysis and "learning your codebase" are, to be charitable, optimistic. What I'm interested in is simple: which one wastes the least of my senior engineers' time by flagging nonsense, while still catching the genuinely dangerous stuff?
To give you a concrete example of what separates a useful finding from noise, here's a snippet from a recent PR where the tools diverged wildly. The issue was a potential use-after-move.
```cpp
std::vector processThings(std::vector input) {
auto filtered = std::move(input);
// ... some logic on filtered ...
if (someCondition) {
// Tool A (Sonar) flagged this correctly: 'input' used after move.
// It understood that 'input' is in a valid but unspecified state.
logSize(input.size()); // BAD
return {};
}
return filtered;
}
```
Tool A and Tool B caught this. Tool C remained silent. Tool D, however, also flagged a dozen other lines in the same file as "suspicious use of moved object," all of which were on completely different variables that were never moved from. That's the noise that erodes trust and leads to people just disabling the bot.
So, my question isn't about which tool has the most features or the prettiest dashboard. It's specifically about **precision on C++ PRs in a real, complex production environment**. Has anyone done a rigorous, apples-to-apples comparison on a non-trivial codebase? I'm talking about:
* The ratio of actionable, correct findings to total findings (precision).
* How they handle modern C++ (17/20) constructs versus older patterns.
* Whether their "security" findings for C++ are actually relevant or just cookie-cutter CVE matching that doesn't apply to the context.
* The configurability of rules without having to write a novel's worth of regex to suppress false positives.
I'll start: in my trial, the leader in precision was the one that was also the most painful to configure, and the worst was the one that bills itself as "zero-configuration." I suspect there's an inverse correlation there. What's everyone else seeing?
-- Cam
Trust but verify.