Walkthrough: Connecting Continue to a self-hosted Ollama instance on a remote server

Establishing a local AI development assistant like Continue is a significant productivity enhancer, but its true potential is unlocked when you connect it to a powerful, self-hosted language model. While many default to a local Ollama instance, developers often have dedicated remote servers with superior hardware for running larger models. The documentation for this specific integration path is often assumed, leaving a non-trivial configuration puzzle involving network layers, authentication, and correct endpoint formatting. This walkthrough will map the complete data flow and necessary configuration to successfully tether the Continue VS Code extension to an Ollama instance hosted on a remote Linux server.

The core challenge is that Continue communicates with Ollama via its HTTP API, which by default listens only on `localhost`. To enable remote access, you must reconfigure Ollama to accept connections from your development machine, while considering security implications. The procedure involves three distinct system layers: the remote Ollama server, the local Continue configuration, and the network in between.

**1. Remote Server: Ollama Configuration & Service Binding**

First, on your remote server (e.g., IP `192.168.1.100`), you must modify the Ollama service to bind to a network interface accessible to your client machine. This is controlled via the `OLLAMA_HOST` environment variable.

- Edit the systemd service file (typically `/etc/systemd/system/ollama.service`) or the service startup script.
- Locate the `[Service]` section and add the `Environment` line:
```ini
[Service]
...
Environment="OLLAMA_HOST=0.0.0.0:11434"
```
- This instructs Ollama to listen on all interfaces, port 11434. For a production scenario, you would restrict this via firewall rules to your specific client IP range.
- Apply the change and restart the service:
```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```
- Verify the service is listening on the intended interface: `sudo netstat -tulpn | grep 11434` should show a line with `0.0.0.0:11434`.

**2. Network Layer: Firewall & Connectivity**

Ensure the server's firewall (e.g., `ufw`) allows inbound traffic on port 11434 from your client's IP address. A basic rule might be `sudo ufw allow from 192.168.1.50 to any port 11434`. Test basic connectivity from your local machine using `curl http://192.168.1.100:11434/api/tags`. You should receive a JSON response listing available models.

**3. Local Machine: Continue Configuration**

Within VS Code with Continue installed, open the Continue configuration file (`~/.continue/config.json` or within your workspace). You must define a custom model that points to your remote Ollama endpoint, specifying the exact API base URL.

```json
{
"models": [
{
"title": "Ollama Remote",
"provider": "openai",
"model": "llama2", // This must match the model name pulled on your remote server
"apiBase": "http://192.168.1.100:11434/v1", // Critical: note the /v1 path
"apiKey": "ollama" // apiKey is required by the OpenAI-compatible client, but Ollama ignores it
}
]
}
```

Key configuration mapping notes:
- The `provider` must be set to `"openai"` because Ollama provides an OpenAI-compatible API endpoint.
- The `apiBase` must precisely be `http://[SERVER_IP]:11434/v1`. The `/v1` suffix is essential, as it routes requests to the correct OpenAI-compatible endpoint.
- The `model` parameter must correspond exactly to a model you have pulled on the remote server (e.g., `"codellama:7b"`).
- The `apiKey` field is structurally required but functionally ignored by Ollama in its default configuration; any string can be used.

**Potential Pitfalls & Verification**

- **SSL/TLS:** This example uses HTTP. For any traffic crossing untrusted networks, you should configure Ollama with TLS. This involves setting `OLLAMA_HOST` to ` https://0.0.0.0:11434` and providing certificates, then updating the `apiBase` in Continue to ` https://`.
- **Connection Refused:** Double-check the server's firewall, the Ollama service status, and that the `OLLAMA_HOST` environment variable is correctly applied (check with `systemctl show ollama --property=Environment`).
- **Timeout or No Completion:** Verify the model name is correct. Use the `curl` test above to confirm the API is responsive. Check server resources (GPU memory) to ensure the model can load successfully.
- **API Version Mismatch:** The `/v1` endpoint is stable, but ensure your Ollama version is relatively recent (>= 0.1.0).

Following this data flow mapping ensures Continue's requests are correctly routed from your IDE, through the network, to the remote Ollama API, and back with completions. This setup optimally utilizes centralized compute resources while maintaining a seamless developer experience within VS Code.