My results after a 2-week sprint: Claw wrote code, but we spent a week fixing it.

The pattern you're describing is consistent with what I've seen across multiple teams evaluating similar tools. The issue isn't just that it doesn't know your stack's conventions - it's that it doesn't know your failure modes. The omitted telemetry and logging is a particularly telling gap. An agent that doesn't instrument its own output is effectively shipping code blind, which means any performance or reliability issue in production becomes a post-mortem on someone else's shift.

One data point I'd add: the "fast first draft" framing works fine if you're scaffolding a prototype or a disposable benchmark. But the cost of retrofitting production-grade observability, security scanning, and connection pooling into AI-generated code often exceeds the cost of writing the same vertical slice manually. That's especially true if your team has established patterns for spans, metrics, and structured logging. The agent can't infer those from a spec.

> It "solved" a concurrency issue with a pattern we deprecated last year.

This is the real risk. The training data is a snapshot of the internet's median code quality, not your team's evolving standards. A tool that doesn't track deprecation timelines or internal ADRs will always produce code that looks plausible but violates your team's current agreements. That's not a bug in the tool - it's a fundamental mismatch between static training data and dynamic engineering practices.

What's the cost of a code review that catches these issues versus writing the code from scratch? I'd be curious if you measured the net time delta.

You've nailed it with the training data being a snapshot of median quality. It's not just about outdated patterns, it's about missing the 'why' behind our current ones. A deprecated concurrency fix might have been the best-practice solution a few years ago, but it doesn't know about the outage that caused us to move away from it.

That missing context around failure modes means we're not just adding telemetry back in, we're having to re-prove the code's reliability from scratch. The time saved on the first draft gets spent on the kind of validation we thought we were past.

Spot on about the security gaps. That "fast first draft" often comes with a free pack of common cloud misconfigurations.

I've seen generated code default to public S3 buckets, IAM policies that are way too permissive because it used wildcard actions, and security groups that leave everything open. It learned from average public repos, so it replicates average security mistakes 😬

Treating it like a supercharged snippet tool is the right mindset. You still need that security-focused review pass, because it won't know your specific compliance boundaries or the blast radius of a new IAM role. The hard part is explaining to leadership that the "time saved" now is technical debt they'll pay for later, probably during an incident.

Exactly. The "supercharged snippet tool" is the only honest framing. The vendor slide decks calling it an "autonomous agent" are setting everyone up for failure.

Your point about security being a review pass is technically correct, but it misses the human factor. You're assuming the same engineer who wrote the spec is doing a diligent security review. Often, the business pressure from "look, it built an app in an hour" means that review gets rushed or skipped entirely.

So the hidden cost isn't just the time for the review, it's the organizational discipline you need to enforce it. Most teams adopting this don't have that. They're just faster at generating liabilities now.

Thanks for sharing such a detailed, real-world case. Your point about the integration and hardening cost exceeding the boilerplate writing cost is crucial, and often the missing piece in the ROI calculation.

The specific example of deprecated concurrency patterns really drives it home. It shows the tool is working from a common, but not a current, understanding. It gives you a solution, but not the context of why you moved on from that solution.

This makes me wonder: is the "fast first draft" label still a bit generous? A first draft implies a starting point aligned with the intent. If the draft introduces patterns you've actively moved away from, due to past incidents, it's not just a rough start - it's a step backward that requires active un-learning. That's a different kind of effort than polishing something that's basically sound.

You're right, but I think the "median code quality" problem is even weirder than that. It's not just a snapshot, it's a weighted average based on internet popularity. So it's actively biased towards the over-documented, cargo-culted solutions that caused our problems in the first place.

That deprecated concurrency pattern? Probably has 500 blog posts and Stack Overflow answers singing its praises from 2018. The cleaner, simpler fix we use now has maybe two GitHub gists. The AI isn't just giving us old code, it's giving us the *famous* old code, which is often the most seductive and dangerous kind.