Okay, I know I'm new here, but I've been watching the conversations around Continue for a few weeks, and I feel like I'm missing something. Everyone seems to be celebrating the 'local first' approach to AI in dev tools as this pure, unadulterated win. But on my team, it's starting to feel like a bottleneck.
We adopted Continue specifically for its local model support. The idea of total privacy and no latency was super appealing. But in practice, the smaller local models we can reasonably run (think CodeLlama 7B variants) just aren't as capable as GPT-4 or even Claude for complex refactors or understanding a sprawling codebase. The suggestions are often close, but not quite right, leading to more time spent correcting than if we'd just used a cloud model with a good privacy agreement.
It creates this weird tension. Do we spend engineering time optimizing local inference, managing GPU memory, and updating model files, or do we just want a tool that helps us ship faster? For our core product work—analyzing user journey drop-offs or prototyping a new feature—the speed of iteration feels more critical than the absolute locality of the data, provided the vendor is trustworthy.
Am I being shortsighted? Is there a setup or model we're overlooking that makes the local experience truly competitive for real-world product development tasks? I'm genuinely curious how other teams are balancing this. The dogma around 'local only' seems to ignore the practical trade-offs in velocity.