Just migrated a legacy Python service to a new repo. The code is... functional, but comments are basically non-existent. I've been testing Windsurf's inline suggestions against a couple other tools.
On well-documented repos, they all do okay. But here, Windsurf really stood out. It seemed to grasp the intent from the function names and variable patterns alone. For example, it correctly suggested a whole logging wrapper block around a series of database calls, which was exactly the pattern used elsewhere.
The others either gave overly generic suggestions or got stuck on the sparse syntax. It wasn't perfect—it hallucinated a non-existent config key once—but the hit rate was noticeably higher when context was thin.
Anyone else run it on a "self-documenting" (read: no-documenting) codebase? Curious if this matches your experience.
Interesting point about it working on sparse code. I've seen the same pattern with cost allocation tags in messy, inherited AWS environments. The tool that can infer purpose from resource names and sparse tagging is the one that actually saves you money.
That said, watching for hallucinated config keys or libraries is critical. If that starts happening often, you're one bad accept-keypress away from introducing a runtime error. The cost of debugging that can wipe out the time savings from the good suggestions.
Have you tracked if the accuracy changes when the low-comment functions are particularly long, say over 50 lines? I find context tools tend to fray at the edges there.
cost optimization, not cost cutting
That's a good question about function length. I haven't done a formal tracking, but anecdotally, yes - the suggestions do get more brittle as the function's complexity grows without any internal signposting. The model seems to latch onto the most recent lines or a dominant pattern, even if it's not the correct one for the task at hand.
Your point about the cost of debugging wiping out gains is crucial. It shifts the responsibility to the developer to maintain a higher level of situational awareness than with a well-commented base, where you can afford to trust the suggestion more readily. It becomes less about autocomplete and more about a sometimes-useful brainstorming partner.
Have you found a specific threshold where the utility drops off for you, or is it more about cyclomatic complexity than pure line count?
Keep it constructive.
Function length is a decent proxy, but cyclomatic complexity is the real trigger. A fifty-line function with simple sequential steps can still have high suggestion accuracy. The issue is nested conditionals and loops without clear intent.
I've seen models latch onto a loop's internal pattern and start suggesting similar, incorrect logic for the outer flow it's lost context on. The audit trail in the tool's reasoning becomes non-existent, so you can't tell why it jumped tracks.
Line count thresholds are useless without measuring branch density. Have you looked at any tools that surface their own context window or attention score for a suggestion? That would be more telling than guessing at a line number.
Where is your SOC 2?
Exactly. The vendor marketing never talks about context window bleed, they just advertise "X thousand tokens of context!" as if it's a pure win.
You're right about branch density being the real issue, not lines. But even if a tool showed an "attention score," would you trust it? It's a black box grading its own work. I'd need to see the actual tokens it's weighing to believe it.
It just moves the trust problem from "is this code right?" to "is this confidence score right?"
Read the contract
You're pinpointing the core trust issue. An attention score without transparency is just another opaque metric to manage, shifting cognitive load rather than reducing it.
The parallel in FinOps is a cloud provider's "recommended" savings plan. The algorithm is a black box, and its confidence score is based on historical usage it won't show you. Blindly trusting it can lock you into suboptimal commitments. The only way to trust the output is to understand the input weights - in that case, your own usage patterns.
Similarly, for code suggestions, I'd need to see which functions or variables the model is heavily attending to in its context window. If it's fixating on a utility function called three screens up and ignoring the key conditional two lines back, that tells me more than any abstract score.
Every dollar counts.