Hey everyone! I've been diving into Continue for the past couple of weeks, mostly for helping with SQL transformations and some Python ETL script reviews. It's been super helpful for catching little bugs and suggesting improvements.
I'm curious though: all the examples and default setup I see point to using OpenAI or Anthropic models. At my company, we have some data governance rules that require using a different cloud provider's LLM service (like Azure's offerings or maybe even Google's Vertex AI). Has anyone actually set up Continue to work with something like that?
I tried poking around the config file, and I see there's a `models` section where you can define a provider, but I'm a bit lost on the specifics. Do you just change the base URL and API key, or are there more steps? I'm worried about compatibility—like, will all the features (code completions, chat, etc.) work the same if the API response format is slightly different?
If anyone has gone down this path, I'd love to hear about your experience! What worked, what didn't, and any config snippets you're willing to share would be amazing. I'm really excited to get this working in our stack, especially to tie it into our Airflow dev workflow.
-- rookie
rookie
Yeah, you can point it at other endpoints. The config's `models` array is the key. For something like Azure OpenAI, you'd define a provider of "openai" but override the `apiBase` to your Azure endpoint and use the corresponding `apiKey`. The model name becomes your deployment name.
The bigger issue is whether the provider's API is truly OpenAI-compatible. Some features, like streaming or certain chat formats, can be brittle if the vendor deviates even slightly. Vertex AI's Gemini API, for instance, has a different JSON structure last I checked. You might spend more time writing an adapter than gaining any value.
It usually works for basic completions. For the full chat experience, you're often at the mercy of the provider's implementation details. I'd start with the simplest use case first before committing to a full rollout.
null
Absolutely, you're on the right track looking at the `models` section. user441's point about API compatibility is crucial - it's the make-or-break factor. I've set this up with Azure OpenAI and it works seamlessly because it's truly OpenAI-compatible. For something like Vertex AI with Gemini, you're likely going to hit snags with the chat interface unless you're only using basic completions.
The specific step you're missing is often the `model` field itself. When you switch to a different provider's endpoint, you need to match the `model` name to exactly what that provider expects. For Azure, that's your deployment name. For others, it might be a full model path. I'd recommend starting with a simple test: configure one model for a non-critical task like documentation generation, and see if the conversation flows correctly. The streaming responses are often where the format differences break.
Also, check the Continue documentation for "custom providers" - there's a community-driven list of adapters for some services that can save you the headache of matching the API schemas yourself. If your company's chosen provider isn't there, be prepared for some trial and error adjusting the request/response format in the config. It's doable, but set aside a couple hours for debugging.
Stay connected