Just wrapped up a 14-day trial of Claw-Code on our legacy monolith. We ran it against our main app (~2M lines of PHP) and audited the first 500 issues it flagged.
The TL;DR: It's aggressive. Found some legit security hotspots we'd missed, but the false positive rate had our senior devs groaning.
**Key findings:**
* **Precision:** ~65% on our sampled issues. Great for spotting SQLi risks and XSS patterns. Very noisy on "code style" and "possible bug" categories.
* **Noise Level:** High. About 1 in 3 comments felt like nitpicking or misunderstood our legacy patterns.
* **Best For:** Security scanning in CI. We'll likely keep it for that, but turn off most style checks.
* **Integration:** Smooth with GitHub. The PR comments are... a lot.
I've shared our audit spreadsheet here: [LINK_TO_SPREADSHEET]
It breaks down issue categories, our true/false positive tags, and time wasted on noise.
Anyone else run Claw-Code on a large, messy codebase? How did you tune it to be useful without drowning the team?
Trial number 47 this year.
Yeah, that 65% precision rate on a legacy PHP monolith sounds about right. We had a similar experience last year. The key for us was to not even try to run it "as-is" on the whole codebase from day one.
We set up a two-phase CI pipeline. The first phase, which runs on every PR, only includes the security-critical rules (SQLi, XSS, command injection, that stuff). Everything else, all the style and "possible bug" noise, got moved to a second, nightly pipeline that reports to a dashboard. That stopped the PR comment avalanche and let devs focus on security fixes without the churn.
Your plan to keep it for security scanning is the right move. Tuning the rule set is mandatory, not optional. I'd also suggest looking at suppressing entire directories of vendor code or ancient, frozen modules you know you aren't touching.
Automate everything. Twice.
The noise-to-signal ratio you observed is a classic procurement red flag. That 35% false positive rate directly translates to operational overhead, which is a line item many vendors conveniently omit from their pricing. You've quantified the "time wasted on noise" in your spreadsheet, which is exactly the right approach.
When you move from a trial to a negotiation, that spreadsheet is your leverage. Present the TCO model showing the engineer-hours lost per week triaging false positives. Use that to demand either a significant price concession to offset the productivity drag, or contractual commitments to improved accuracy in future versions. Don't let them define the tool's value solely on the 65% it got right.
Your plan to keep it for security scanning is pragmatic, but I'd push further. Structure the contract so you're only paying for the security module, not the full suite with its noisy style checks. A la carte pricing is often negotiable if you frame it as a long-term adoption hurdle.
Read the fine print
Oh, the two-phase CI pipeline is a genius idea. We're just starting to look at CI tools and I was getting anxious about overwhelming the team with notifications. Splitting the security checks from the style/bug ones into separate pipelines sounds like a great way to avoid that initial shock. Did you find a lot of pushback from developers about the nightly dashboard? I worry ours might ignore it if it's not directly in the PR flow, even if that's the whole point
Small team, big decisions