Has anyone run a proper test of Claw-Code on legacy Java monoliths? Need actual data.

data_diver_43 · 2026-08-22T17:56:45Z

Hey everyone, been lurking here for a bit. I'm a junior analyst, but my team got handed this massive, old Java monolith (we're talking Java 8, some parts even older, with tons of custom frameworks). We're trying to improve code quality and security, and management is pushing to integrate an AI code review tool into our PR process. We've been demoing a few, and **Claw-Code** keeps coming up. Their marketing talks a big game about handling complex, legacy code. But I'm skeptical—most demos use clean, modern examples. Before I push for a trial, I need to know: **has anyone actually run a systematic test of Claw-Code on a real legacy Java codebase?** I'm looking for concrete data, not just feelings. Specifically: - **Precision/Recall:** How many of its findings were actual, actionable issues vs. noise (false positives) on *legacy* patterns? Did it miss major security flaws or bad patterns common in old Java (e.g., resource leaks, null pointer risks, outdated APIs)? - **Comment Quality:** Were its suggestions actually applicable? Legacy code often can't just adopt a modern library or refactor massively due to dependencies. Did it suggest plausible, incremental fixes? - **Review Queue Impact:** How much did it slow down your senior devs? If it flags 200 issues per PR, that's a non-starter. I've tried to set up a small test myself on a nasty service module. Here's a snippet of the kind of code it's facing: ```java public class LegacyService { private static HashMap pool; // static mutable map, yikes public Data fetch(String id) { Connection conn = pool.get(id); // no null check, potential NPE Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("SELECT * FROM data WHERE id=" + id); // concatenation, SQL injection risk // ... and often no close() calls in finally blocks } } ``` A good tool should catch the SQL injection, the missing null check, the resource management problem, and maybe flag the static mutable collection. But will it also drown us in false positives about using `HashMap` instead of `ConcurrentHashMap` where the context doesn't need it, or complain about missing `@Override` annotations from a time before that was standard? If you've run tests, what was your setup? Did you compare it against other tools (like SonarQube with new AI features, or CodeRabbit) on the same codebase? Any hard numbers would be incredibly helpful.

Page 3 / 3 Prev

AI Code Review Tool Comparisons

Last Post by Integration Ian 1 month ago

31 Posts

31 Users

0 Reactions

65 Views

RSS

Integration Ian

(@integration_ian)

Honorable Member

Joined: 5 months ago

Posts: 396

23/08/2026 5:51 pm

You've already gotten the data you asked for in this thread, summarized pretty well in the last few posts. The precision numbers are abysmal for a legacy monolith with custom frameworks - expect 10-15% on a good day.

The comment quality is the real killer. It can't suggest incremental fixes because it doesn't understand your constraints. We saw it repeatedly tell teams to replace a perfectly fine, thread-safe `StringBuffer` with `StringBuilder` just to hit a modern checkbox, introducing a subtle bug.

Before you even consider a trial, get a sample of your worst, most framework-heavy module and run it through their free scanner. The output will be hundreds of lines of warnings about your own architectural choices. Show that report to management as the "concrete data" for why this is a waste of engineering time. You'll spend months training the tool to ignore your codebase.

Integration is not a project, it's a lifestyle.

ReplyQuote

Page 3 / 3 Prev