Hey everyone, I've been digging into using both ChatGPT (GPT-4) and Claude (Sonnet 3.5) for reviewing our team's code at my SaaS startup. We're a small team of 5 devs, and we've been trying to offload some of the initial review burden to these assistants to speed things up. Overall, they're impressive, but I've hit some very specific, reproducible failures that are worth sharing.
**The Prompt:** "Review this Python function for potential bugs and edge cases. It's meant to parse user-provided date strings from our analytics dashboard."
```python
def parse_user_date(date_str, default_date=None):
from datetime import datetime
try:
return datetime.strptime(date_str, '%Y-%m-%d')
except ValueError:
return default_date
```
**Claude's Output (The Failure):** Claude confidently stated: "This function looks good, but consider adding validation for timezone-aware datetime objects. The format `%Y-%m-%d` might not parse strings like `'2024-01-01+00:00'`. Also, the `default_date` parameter should be type-checked to ensure it's a `datetime` object." It then provided a refactored version with `pytz` and an `isinstance` check.
**The Problem:** This is a classic case of hallucinating requirements. Our function's spec is for simple, naive date strings from a form field. The suggestion about timezone-aware parsing is completely unnecessary for our use case and adds needless complexity. More critically, the `default_date` is meant to be *any* default return value (like `None` or a string placeholder), not necessarily a `datetime` object. Claude's "improvement" would actually break our existing code in several places.
**The Correct Review:** A helpful review would have noted:
* The real risk: `date_str` itself could be `None`, causing `strptime` to raise a `TypeError`, not a `ValueError`.
* The `default_date` is returned as-is, which might be fine, but could lead to inconsistent return types (sometimes `datetime`, sometimes `None` or a string). That's the actual design question to flag.
* The import is inside the function, which is fine for a simple utility, but worth a small performance note.
**Takeaway:** Both assistants often fail by inventing edge cases or "best practices" that don't apply, making the review less about *your* code and more about a generic, overly-cautious checklist. For a startup moving fast, these "improvements" can actually introduce bloat or new bugs.
Has anyone else run into similar issues where the assistant's "thoroughness" just leads you down the wrong rabbit hole? What's your process for getting genuinely useful reviews from them?
Cheers!
Automate the boring stuff.