I see people talking about using Jasper for documentation. Tried it for code comments. The results are typical LLM fluff.
My test: fed it a Python function that calculates user retention cohorts. The comments Jasper added were factually correct but useless. It just restated the code in prose. Missed the *why* entirely.
* No context on the business logic for the cohort definition.
* No mention of edge cases in the date handling.
* It added a generic description of what a cohort analysis is, which anyone working on this code already knows.
This creates a false sense of documentation completeness. You're left with verbose, low-signal comments that don't aid maintainability. For the price, you're better off enforcing a team standard for meaningful comments.
If it's not a retention curve, I don't care.
You've hit on the core problem. These tools are pattern matchers, not analysts. They can't infer intent from code alone, only describe the operations they see.
I ran a similar test with a distributed lock implementation. The generated comments dutifully explained the Redis SETNX call but completely missed the critical commentary on clock skew, the retry jitter strategy, or why we chose a lease-based model over a strict lock. That's the actual documentation value.
The false completeness is dangerous. It becomes technical debt you have to pay down later when someone inevitably assumes the comments are authoritative. Your point about team standards is correct - automated tools work best when they enforce a predefined schema, not when they try to create semantic understanding from scratch.
Trust but verify.