Tutorial: Building a simple RAG pipeline with Le Chat and local embeddings.

Everyone's rushing to build RAG pipelines with the usual API suspects, costing a fortune in vector DB credits and embedding calls. Let's be contrarian: use Le Chat for the LLM and run everything else locally. It's more resilient, cheaper, and you might actually learn how the sausage is made.

We'll use `sentence-transformers` for local embeddings, `chromadb` as our vector store, and Le Chat's "Mistral Large" model as the reasoning endpoint. The goal is to ask questions about your own documents without your data leaving your machine, except for the final prompt to Le Chat.

First, the local embedding and indexing script. This assumes you have a `documents/` folder with some text files.

```python
# ingest.py
from sentence_transformers import SentenceTransformer
import chromadb
from chromadb.config import Settings
import os

# Initialize models and clients locally
embed_model = SentenceTransformer('all-MiniLM-L6-v2')
chroma_client = chromadb.PersistentClient(path="./chroma_db", settings=Settings(anonymized_telemetry=False))
collection = chroma_client.get_or_create_collection(name="docs")

# Read and chunk documents
docs, metadatas, ids = [], [], []
for filename in os.listdir("documents"):
with open(os.path.join("documents", filename), 'r') as f:
text = f.read()
# Simple chunking - you'll want something better for production
chunks = [text[i:i+500] for i in range(0, len(text), 500)]
for i, chunk in enumerate(chunks):
docs.append(chunk)
metadatas.append({"source": filename})
ids.append(f"{filename}_{i}")

# Generate embeddings locally and store
embeddings = embed_model.encode(docs).tolist()
collection.add(embeddings=embeddings, documents=docs, metadatas=metadatas, ids=ids)
print(f"Indexed {len(docs)} chunks.")
```

Now, the query pipeline. This is where Le Chat comes in, but only with the relevant context we feed it.

```python
# query.py
import chromadb
from chromadb.config import Settings
from sentence_transformers import SentenceTransformer

def query_le_chat(question, context):
# This is the conceptual step. You'd use the Le Chat API here.
# Construct a prompt with the retrieved context.
prompt = f"""Use the following context to answer the question.
If the context doesn't contain the answer, say so.

Context:
{context}

Question: {question}
Answer:"""
# In reality, you'd call `client.chat.completions.create` with the Le Chat endpoint.
# For now, we'll print the prompt structure.
print("Prompt to Le Chat would be:")
print(prompt[:500] + "...")
# The actual LLM call happens here.
return "[Le Chat's generated answer would appear here]"

# Local components
embed_model = SentenceTransformer('all-MiniLM-L6-v2')
chroma_client = chromadb.PersistentClient(path="./chroma_db", settings=Settings(anonymized_telemetry=False))
collection = chroma_client.get_collection(name="docs")

# Query
question = "What is the capital of France?"
question_embedding = embed_model.encode([question]).tolist()
results = collection.query(query_embeddings=question_embedding, n_results=3)
retrieved_context = "n---n".join(results['documents'][0])

# Send only the question and retrieved context to Le Chat
answer = query_le_chat(question, retrieved_context)
print(answer)
```

The irony is delicious. You're using a state-of-the-art model like Mistral Large through Le Chat, but you've sidestepped their embedding API and avoided a managed vector database. Your only billable call is the final chat completion. The pipeline will survive an API outage for everything but the final answer, and you can swap the LLM endpoint with minimal fuss.

Is this "best practice"? Probably not according to the all-in-one platform vendors. But it works, it's transparent, and it doesn't require a credit card to prototype. The next time your cloud vector store has a latency spike, remember this little local experiment.