I'm helping my team evaluate AI code review tools. We've tried a couple that flagged a ton of obvious non-issues, and the noise just made everyone ignore them. I'm skeptical of any tool that claims "high precision" without showing the data.
What are the top tools you've used that actually have a low false positive rate? I'm especially interested in real-world experience on Salesforce Apex and some JavaScript. How often do you find yourself dismissing their suggestions as irrelevant?
You're right to be skeptical about unsubstantiated precision claims. In our A/B testing of review tools on a mixed codebase including JavaScript, we measured false positive rates by logging every dismissed suggestion as an event and analyzing the cohorts. The tools with the lowest rates, around 10-15%, were those that allowed granular rule configuration from the start, not just on/off toggles.
For Salesforce Apex specifically, the static analysis landscape is different. Tools built for general languages often struggle with the platform's unique patterns. We found the lowest noise came from specialized tools like PMD with the Apex rule set, but only after we meticulously disabled rules for governor limit warnings on trivial operations, which were constant false positives.
This table from our 90-day pilot might be useful:
| Tool | JS FP Rate | Apex FP Rate | Key Mitigation |
|------|------------|--------------|----------------|
| SonarQube (Custom Profile) | 12% | 28% | Required heavy rule suppression |
| CodeClimate (Eng) | 18% | N/A | Good out-of-box JS defaults |
| PMD (Apex focused) | N/A | 15% | Disables trivial limit checks |
The real metric is dismissals per hundred lines of code. For our team, anything above 0.2 led to tool abandonment within two weeks. The configurability overhead is a necessary cost.
Data > opinions
Thanks for sharing those numbers, that's super concrete. The dismissal logging is a great idea.
> only after we meticulously disabled rules for governor limit warnings on trivial operations
I hit this exact thing in a recent pipeline script. The out-of-the-box Apex rules flagged a SOQL query in a loop that only runs once during a daily sync, so it's not a real governor risk. Took a while to find the right suppression.
For our Python ETLs, we started with a basic linter profile and the false positives were overwhelming. Did you build your custom SonarQube profile iteratively, or start from scratch?