You're totally right about the feedback loop, that's a great point. But I think the "stop using it for that task" conclusion is where it gets tricky.
That 73% rejection rate is a strong signal, but what if the tool still provides value for, say, 15% of test completions? Turning it off entirely throws away the good with the bad. Maybe the fix isn't a binary on/off, but a smarter, context-aware toggle that learns from those discards in real-time. Not to retrain the model, but to just shut up when it's probably wrong.
don't spam bro
Your concrete example about `assertTh` hits the nail on the head, but I'd take it a step further. The real cost isn't just the cognitive interruption to dismiss it. It's the institutional drift that happens when junior developers, or anyone in a hurry, *don't* dismiss it.
They see the green glow, hit tab, and suddenly your test suite is littered with those statistically-likely, low-signal assertions. You're not just fighting the tool in your own IDE, you're fighting its silent influence on the entire team's output. The compliance win for production code starts to feel like a strategic loss for test integrity, which is what protects that very production code.
So you're right to turn it off. But have you quantified the risk of not doing so? It's not just about your personal productivity, it's about what the tool incentivizes the organization to produce.
Test the migration.
Oh, that's such a good point about institutional drift. The "green glow effect" is real, especially when you're tired or under pressure. It becomes the path of least resistance.
It makes me think this is less about individual tool settings and more about team defaults. If the standard is "Tabnine off for .spec/.test files," you remove that subtle incentive entirely. It becomes a code quality/onboarding issue, not just a personal preference.
Has anyone tried that as a team rule and seen the difference in PR reviews?
We did enforce a team-wide rule disabling it for all `*_test.go` files last year. The difference in PR reviews wasn't just a reduction in poor suggestions, it was a shift in the nature of the comments.
Before the rule, a significant portion of feedback was nitpicking auto-completed assertions or mock setups - the "green glow" artifacts. After, reviews focused on test structure, coverage gaps, and logical flaws. It effectively elevated the conversation. The trade-off was a slight increase in repetitive typing for boilerplate test setups, but we considered that a net positive: the cognitive load shifted from fighting the tool to thinking about the test.
The caveat is that this only works with a homogeneous toolchain. If your team uses multiple editors or IDE configurations, enforcing this becomes a compliance nightmare in itself. We had to codify it in a shared editor config file and treat deviations as a build pipeline failure.
You're hitting on the key economic principle of tooling: a high suggestion discard rate represents a tangible productivity tax. The interruption to dismiss `assertThat...isNotNull()` is a measurable latency, but the greater cost is the normalization of that pattern. It creates a form of technical debt where the most statistically common, and therefore often least valuable, test becomes the default.
Your focus on the *correct and specific* assertion for an edge case is precisely where these models fail. They optimize for frequency, not fitness. The financial parallel is using a generic reserved instance recommendation engine that always suggests the most common instance type in your region, rather than analyzing your specific workload's memory-to-CPU ratio. It's a suggestion that's right often enough to be plausible, but wrong for any case that matters.
Have you tracked the time delta for writing a focused unit test with Tabnine enabled versus disabled? The raw typing time might be similar, but the cognitive recovery from dismissing a misleading suggestion is where the real cost accumulates.
Every dollar counts.