Hey folks! Been seeing a lot of buzz about Le Chat (Mistral) for internal knowledge Q&A, so I decided to wire it up to our team's Postgres DB that holds our runbooks and deployment docs. My usual stack is Prometheus/Grafana for alerts, but for this text-based querying, Le Chat seemed like a fun experiment.
The goal was simple: let the team ask questions in plain English like "What's the rollback procedure for service X?" and get answers based on our actual documentation. Here's the step-by-step I followed, focusing on the monitoring/ops mindset we all love:
**Step 1: Prep your schema**
I created a separate table to store the documentation chunks with metadata. This makes the embeddings step cleaner.
```sql
CREATE TABLE internal_docs (
id SERIAL PRIMARY KEY,
content TEXT,
source_url VARCHAR(255),
service_name VARCHAR(100),
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```
**Step 2: Generate embeddings via Mistral's API**
I used the `mistral-embed` model to create vectors for each text chunk. The key here is batching your inserts to avoid timeouts. I wrote a small Python script that:
- Fetches rows from `internal_docs`
- Calls the embedding API in batches of 50
- Stores the vector in a new `pgvector` column
**Step 3: Set up the Le Chat connection**
Le Chat's platform has a "Connections" section. I selected PostgreSQL and provided the connection details (host, db name, user). **Crucial:** Use a dedicated, restricted user with read-only access *only* to the necessary tables/views. Never use admin credentials! 🚨
**Step 4: Configure the RAG pipeline**
Within Le Chat's interface, I mapped the query to perform a cosine similarity search on the embeddings column. The response is built from the top 3 matching chunks. I also added a filter to only search docs updated in the last year to avoid stale info.
The initial results are promising! The model correctly pulls steps from our runbooks. However, I'm now thinking about **alerting** on this setup—maybe a cron job to validate the connection daily and post to a Slack channel if the embeddings job fails. Has anyone else set up monitoring for their AI-powered Q&A systems? I'd love to compare notes on what metrics you track (latency, hit rate, user feedback scores).
If it's not monitored, it's broken.
Your focus on schema design for the embedding source is critical. Too many implementations skip that step and try to vectorize raw production tables, which invariably leads to permission issues and stale data.
One nuance you might consider for your Python batch process: logging each batch's success/failure rate and timestamp to a separate monitoring table. This gives you observability into the embedding pipeline itself. You can then tie failures to specific API error codes or content length, allowing for automatic retry logic on certain conditions.
Also, depending on your documentation volume, you may want to pre-compute and store the embeddings directly in a dedicated Postgres column using the pgvector extension. This eliminates the API call latency for repeated queries and allows you to version your vectors alongside the source text, which is useful for auditing change over time.
trust but verify
Okay, the batching idea for the embeddings API is super smart to avoid timeouts. That's something I'd have missed for sure.
When you say you call the API in batches, are you handling retries with exponential backoff? I tried something similar last week with a different service and got absolutely throttled - had to add a pretty aggressive sleep between batches. My script looked like it was working, then just blew up after 100 requests 😅
Also, curious if you thought about chunking the `content` text before embedding? My team's runbook entries can be huge, and I've read that can mess with embedding quality.
null