Hey everyone, I've been trying to level up my SQL skills and wanted to share a project I just finished at my company using Cartesia. We needed a way to automatically flag non-compliant language in customer support chat logs.
The basic idea was to scan incoming chat messages against a dynamic rules table we maintain (things like prohibited phrases, specific compliance keywords). I used Cartesia to run a similarity search between each chat message and our rules library. The cool part was setting a confidence threshold so it only flags the high-probability matches for review.
Here’s the core query structure I landed on after a lot of trial and error:
```sql
WITH message_embeddings AS (
SELECT
chat_id,
message_text,
cartesia.embed(message_text, model => 'text-embedding-3-small') AS embedding
FROM support_chats
WHERE chat_date = CURRENT_DATE
),
rule_embeddings AS (
SELECT
rule_id,
rule_text,
cartesia.embed(rule_text, model => 'text-embedding-3-small') AS embedding
FROM compliance_rules
WHERE rule_active = TRUE
)
SELECT
m.chat_id,
m.message_text,
r.rule_id,
r.rule_text,
cartesia.similarity(m.embedding, r.embedding) AS match_score
FROM message_embeddings m
CROSS JOIN rule_embeddings r
WHERE cartesia.similarity(m.embedding, r.embedding) > 0.85
ORDER BY match_score DESC;
```
The results get dumped into a Tableau dashboard for the compliance team. It’s cut down their manual review time by about 70% so far 😅
I’m still pretty new to this, so I’d love some feedback. Has anyone else built something similar for text monitoring? I’m wondering if I should be pre-filtering the CROSS JOIN somehow, as the rule list grows it might get slow. Also, any tips on tuning the similarity threshold? We started at 0.9 but missed some paraphrased violations.