Skip to content
Notifications
Clear all

My results after 30 days of using Tabnine Pro for Go development

8 Posts
8 Users
0 Reactions
59 Views
(@gregoryp)
Reputable Member
Joined: 4 months ago
Posts: 257
Topic starter   [#28849]

Having spent the last month rigorously evaluating Tabnine Pro within my primary development environment—focusing on a mid-sized Go codebase (~85k LOC) for a Kubernetes operator—I have compiled a detailed assessment of its impact on my workflow, code quality, and overall productivity. My methodology involved tracking specific metrics for the first and last weeks, alongside qualitative observations.

**Primary Configuration & Environment:**
- **Editor:** Neovim with the Tabnine plugin (`coc-tabnine`).
- **Project:** Go 1.21, heavily utilizing `context`, `client-go`, and custom resource definitions.
- **Tabnine Pro Settings:** Enabled full-line completions, activated automatic code refactoring suggestions, and left the default model size.

**Quantitative Results (Averages per coding session):**
- **Completion Acceptance Rate:** Week 1: ~28%. Week 4: ~42%. The increase correlates with the model better learning the project's patterns.
- **Keystrokes Saved (estimated):** Approximately 35-40% on boilerplate and idiomatic Go structures (e.g., error handling, struct initialization, loop patterns).
- **Most Impactful Completions:**
- Method chains on `client-go` objects (e.g., `.Namespace("default").List(ctx, metav1.ListOptions{})`).
- Error wrapping patterns with `fmt.Errorf` and `%w`.
- Standard `http.Handler` implementations and middleware stubs.
- **Least Useful Completions:** Complex business logic inside controller reconciliation loops. Tabnine would often suggest plausible but contextually incorrect control flows.

**Qualitative Analysis & Key Observations:**

* **Strengths in Idiomatic Go:**
Tabnine excels at generating code that adheres to Go conventions. It reliably suggests:
- Proper `if err != nil` handling and early returns.
- Struct literals with field names.
- WaitGroup, Mutex, and channel patterns with correct zero-value usage.
- Unit test table-driven structures.

```go
// Example: After typing `wg := &sync.WaitGroup{}`, Tabnine frequently suggested the pattern:
for _, item := range items {
wg.Add(1)
go func(i SomeType) {
defer wg.Done()
// process i
}(item)
}
wg.Wait()
```

* **Integration with Project Context:**
The model's ability to leverage open files and project-specific terminology improved noticeably after several days. It began accurately completing custom type names and function signatures from our internal packages.

* **Performance & Latency:**
The local model (running via the Tabnine daemon) introduced no perceptible editor latency. Full-line completions appear within 200-300ms, which is acceptable but occasionally disrupts flow when incorrect.

* **Notable Pitfalls & Limitations:**
- **Over-reliance Risk:** It can generate syntactically valid, logically flawed code, especially around concurrency. Careful review remains mandatory.
- **Configuration Blindness:** It does not comprehend project-specific `go.mod` replace directives or build tags, sometimes suggesting imports from modules not in use.
- **Limited Understanding of K8s Semantics:** While it completes client-go method chains, it lacks deeper knowledge of Kubernetes semantics (e.g., suggesting correct Finalizer names or understanding OwnerReferences).

**Conclusion on Value Proposition:**
For a Go developer working on cloud-native tooling, Tabnine Pro provides tangible efficiency gains in the "plumbing" and boilerplate aspects of coding. It acts as a powerful accelerator for idiomatic patterns but does not replace the need for deep domain expertise, particularly in areas like security or complex system design. The cost is justifiable if you spend significant hours in direct coding, but its utility diminishes for architects or those primarily in design/meeting roles. I plan to continue the subscription for another quarter to assess long-term adaptation and will be testing its Terraform/HCL suggestions in a separate evaluation.


infra nerd, cost hawk


   
Quote
(@data_pipeline_guy)
Reputable Member
Joined: 8 months ago
Posts: 388
 

Keystrokes saved isn't the metric I worry about. It's keystrokes *thought about*. If the tool starts suggesting a complex client-go method chain, I still have to stop and read it to verify it's correct. That's a context switch.

I've found these things are great for the boilerplate you mentioned, but for actual logic, they just become a distraction I have to constantly dismiss.

How many of those accepted completions were just filling in struct fields or writing another `if err != nil`? That's what it always comes down to.


SQL is enough


   
ReplyQuote
(@danielp)
Estimable Member
Joined: 4 months ago
Posts: 200
 

That's a great point about the mental tax. I had a similar feeling with GitHub Copilot when it first came out - the constant stream of longer suggestions was actually breaking my flow more than helping.

With Tabnine specifically, I found tuning the suggestion length down helped a lot. It became more of a supercharged autocomplete for the next few tokens rather than trying to write whole functions. That reduced the "need to verify" overhead because the suggestions were more predictable, like finishing a method name I'd already started typing.

But you're right, the real value is in the boilerplate. For my Go work, if it saves me from looking up the exact field names for a `client-go` `ObjectMeta` struct for the 100th time, that's a win. The logic is still entirely mine. Maybe the metric should be "minutes saved not switching to the browser or another IDE window" instead of keystrokes?



   
ReplyQuote
(@finops_tracker_99)
Reputable Member
Joined: 8 months ago
Posts: 273
 

I like the idea of measuring "minutes saved not switching to the browser." That's the real friction cost. For me, it's the same when writing CloudFormation or Terraform - remembering the exact property syntax for a less-used resource.

But there's a hidden cost too: sometimes the suggestion is so close to right, but subtly wrong for my specific use case (like an AWS IAM policy action). Accepting it without that browser check creates a subtle bug I have to find later. So the saved context-switch minutes can get wiped out by debugging minutes if I'm not careful.

I've started treating these tools strictly as a faster way to pull up a reference, not as a source of truth. I still have to validate.



   
ReplyQuote
(@hiker42)
Reputable Member
Joined: 4 months ago
Posts: 232
 

The completion acceptance rate jump is the key metric there, from 28% to 42%. That's the tool adapting to your specific codebase, not just generic Go. It means you're building institutional knowledge into the editor.

But you need to audit what's in that accepted 42%. The risk is it gets very good at suggesting your team's existing patterns, even the bad ones. I'd want to see if those refactoring suggestions ever flagged any of your own repetitive code that the model learned to mimic. Otherwise you're just cementing technical debt faster.



   
ReplyQuote
(@alexh3)
Reputable Member
Joined: 4 months ago
Posts: 254
 

That's a crucial point. The adaptation is a double-edged sword. In my case, the refactoring suggestions did occasionally flag repetitive patterns, like suggesting I extract a common validation block used across three different reconciler methods. But you're right, it's passive - it won't critique architectural patterns.

The real risk I observed was with error handling. Our codebase has a few legacy functions that return `(value, error)` instead of the more idiomatic `(value, bool, error)`. Tabnine started eagerly suggesting that older pattern in new contexts simply because it was prevalent in the trained files. I had to consciously reject it, which felt like fighting the tool's "learning." It wasn't cementing debt so much as offering to rebuild it in new places.


Data is the source of truth.


   
ReplyQuote
(@devops_barbarian_v3)
Honorable Member
Joined: 7 months ago
Posts: 403
 

Exactly. That 42% is a loyalty program for your worst habits. Seen it happen with helm templates - it learns your janky `.Values` nesting and starts suggesting it everywhere. You're not just automating work, you're automating debt issuance.

The refactoring suggestions are basically linters with extra steps. If your team doesn't have the discipline to fix the bad pattern when the linter flags it, a "suggestion" won't change anything. It just becomes background noise you learn to ignore, like a deprecation warning you've seen for six months.

Only way it's useful is if you treat its suggestions like a hostile code review. If it *stops* suggesting your old crap, that's when you know you've actually improved something.



   
ReplyQuote
(@adamk)
Reputable Member
Joined: 4 months ago
Posts: 253
 

Love the data-driven approach, especially tracking acceptance rate over time. The jump to 42% shows it's really learning your codebase.

But I'm curious about one thing in your metrics. You mention keystrokes saved on boilerplate. Does that translate to actual time saved, or is there a mental lag from reviewing the suggestions? For me, the time benefit only clicks when it nails a whole client-go method chain without me having to look up the docs.

The adaptation is great, but watch out for it reinforcing any odd patterns you might have in that 85k LOC. It can be a bit too eager to replicate the past.


Always optimizing.


   
ReplyQuote