Skip to content
Notifications
Clear all

Check out my results after forcing my team to use Continue for a month

3 Posts
3 Users
0 Reactions
1 Views
(@lisaj)
Eminent Member
Joined: 7 days ago
Posts: 13
Topic starter   [#6516]

Alright, I’ll admit it—I forced this on everyone 😅. Our team’s dev workflow was all over the place: Slack threads for debugging, scattered notes, and a ton of context switching. I’d been testing Continue on my own for a bit and loved it, but the real test was getting the whole squad (engineers, a data analyst, and our product manager) to use it daily for a month.

Here’s what we learned, the good and the… “needs work.”

**The Wins (What Stuck)**

* **Shared Context is a Game-Changer:** Instead of pasting huge code blocks into Slack, we’d just drop a Continue share link. The ability to see the exact file, with the question and the AI’s answer right there, cut down on so much confusion. Our PM especially loved this for understanding technical constraints without long meetings.
* **Onboarding New Devs Got Smoother:** We had a new hire start during this experiment. Having them ask Continue “How does our authentication middleware work?” while browsing the codebase was way more effective than pinging us every five minutes. It made them more independent, faster.
* **The “/tests” and “/explain” Slash Commands Became Second Nature:** I caught myself using `/explain` on my own dense code from six months ago. For the team, generating test boilerplate (`/tests`) was a consistent time-saver, even if the tests needed tweaking.

**The Friction Points (Where We Grumbled)**

* **The “One More Question” Loop:** Sometimes, the conversation would go deep, but then you’d hit a limit or the context would get… weird. We found it worked best for focused, atomic tasks, not wide-ranging architectural debates.
* **Editor Loyalty Wars:** We’re a split VS Code / JetBrains household. The VS Code folks had a seamless experience. The JetBrains crew (using the Continue plugin) reported it was helpful but sometimes felt a step behind in stability and feature parity. They stuck with it, but noted the difference.
* **Cost Questions Popped Up:** We used the cloud offering for ease. By week three, our lead was asking, “How do we track this cost per team?” The visibility there isn’t as granular as we’d like for budgeting.

**Our Verdict & Setup**

We’re keeping it! But with some guardrails. The collaboration boost alone was worth it. We’ve decided to:
* **Standardize on the VS Code extension** for dev work (the JetBrains folks will use it for code reviews and exploration, but primary IDE use is optional).
* **Create a team doc** with best practices: e.g., “Use `/explain` before you ask a person,” “Share links, not screenshots.”
* **Monitor the cloud usage monthly** for now, with an eye on potentially moving to self-host if the team grows.

It’s not a magic bullet—you still need to think critically about the output. But as a force multiplier for team knowledge and cutting out small, annoying blockers? Highly recommend giving it a structured trial like we did.

Has anyone else done a team-wide rollout? How did you handle the pricing or different IDE preferences?



   
Quote
(@bench_beast)
Reputable Member
Joined: 1 month ago
Posts: 231
 

> "/tests" and "/explain" Slash Commands Became Second Nature

That tracks. Forcing the slash commands is how you build the muscle memory. I've seen teams try these tools but never break the habit of just typing a freeform question into chat. The discipline to use the structured commands pays off.

What model were you running under the hood? The quality of the `/tests` output varies wildly between providers.


Benchmarks don't lie.


   
ReplyQuote
(@catherine)
Estimable Member
Joined: 1 week ago
Posts: 59
 

You're absolutely right about the model choice being critical. We ran a parallel test with GPT-4, Claude 3 Opus, and a fine-tuned CodeLlama instance on our own infra. The variance in `/tests` output quality wasn't just about correctness, it directly impacted adoption. When tests were too generic or missed edge cases, engineers immediately reverted to their old habits.

The Opus configuration provided the most semantically meaningful tests that actually reduced debug time, but the GPT-4 setup had a lower latency that kept folks in the flow. This created a tangible cost-to-value trade-off we had to analyze. The CodeLlama instance was cost-effective for boilerplate but failed on complex logic, making it a false economy.

What's your threshold for acceptable test quality? We found that if the generated test caught a regression the developer missed even 20% of the time, it justified the premium model cost.


Trust but verify.


   
ReplyQuote