Everyone’s raving about Codeium’s free tier. So I logged 100 hours across various projects, tracking only one thing: how often its suggestions were actually useful. Not just accepted—useful.
Result? 18% acceptance rate. And half of those were just boilerplate or imports. For actual logic or bug fixes? Single digits. Tried it on a slightly niche stack (Elixir/Phoenix) and the suggestions were comically outdated. The security scanning flagged my own test fixtures as “vulnerable code.” Support ticket took 8 days for a canned response. So much for the GitHub Copilot killer.
Open-source tools aren’t perfect, but at least you can fix the nonsense yourself. This feels like another vendor locking you into their mediocre model with a “free” carrot. Privacy policy is a novel if you care about your code. —aB
—aB
Your 18% useful acceptance rate lines up with what I see a lot of people quietly reporting. The key detail you added is breaking it down into boilerplate vs. logic. That's where the real value assessment happens.
The niche stack point is critical. These tools train on the mainstream, so anyone off the beaten path gets a worse experience. It's not just Codeium, it's a model training data problem.
Eight days for a canned response on a security false positive is unacceptable, full stop. That's a vendor readiness issue, not just a model quality one.
—AF
Yeah, that vendor readiness issue is a huge red flag. A slow, canned response to a security false positive can erode trust faster than a mediocre model. It makes you wonder about their internal prioritization.
The niche stack problem is real, but I've even seen it struggle with newer features in mainstream frameworks. If your React codebase uses a lot of the newer hooks patterns, the suggestions can feel a generation behind. It really highlights the training data gap.
Your point about quietly reporting is spot on. Most teams I talk to have similar numbers but don't post them publicly. They just stop using it.
Happy customers, happy life.
Eight days for a canned response isn't a prioritization problem, it's a structural one. Their support can't scale with their marketing.
The quiet abandonment you mention is the real metric. Teams don't file tickets, they just stop using it. That's a death spiral for a tool that needs usage data to improve.
It's not just niche stacks. Try getting a correct suggestion for TanStack Query v5 right now. You won't. The model is always chasing last year's patterns.