Your approach is correct for a bug that's isolated to a single file. Using the chat to find and understand the issue first is the smart move, especially when you're still building context.
The risk comes later, when the fix isn't isolated. I've seen it generate a perfect local solution that broke something three modules away because it didn't understand a shared dependency. For a NaN bug, you're probably safe. For anything touching state or shared logic, you still need to be the one checking the broader impact.
So chat to find, direct edit to fix, but never skip your own integration test.
—AF
Absolutely. It's that disconnect between the formal spec and the runtime reality that gets me. I ran into this last week with a webhook integration. Windsurf suggested a clean fix to validate a payload against our API contract. The code was flawless, but it didn't know that one of our major partners has a bug in their system where they send a `user_id` as a stringified number, even though our spec clearly states integer. The "correct" validation would have broken their integration for weeks before they'd notice.
So now, my first draft from the tool always gets an extra layer: a compatibility shim or a logging flag for those undocumented handshakes. It's less about writing the code and more about encoding those hidden agreements.
null
Oh man, this hits home. We had a similar thing with a payment processor's webhook. The spec said dates were ISO 8601, but they'd occasionally send a Unix timestamp string. The "correct" validation would've silently dropped payments.
I've started treating those first AI drafts as the "textbook" solution, then manually adding the "production" layer. It's like a compatibility map. I'll even add a metric tag in Datadog for those shimmed validations, so we can track when a partner finally fixes their bug and we can sunset the workaround.
> encoding those hidden agreements
That's exactly it. The tool writes the contract, but we have to write the handshake.
Dashboards or it didn't happen.
Exactly. And that's why this whole "pair programming" framing is a bit rich. You're not pairing with a programmer. You're pairing with someone who's only ever read the textbook.
Your payment processor example is perfect. The real engineering happens in those compatibility layers, and the AI has no clue they exist. It gives you a pristine, correct solution to a problem that doesn't actually match reality.
So you end up doing the real work anyway, which is stitching the ideal world to the messy one.
Your vendor is not your friend.