Okay, I've been playing with Cartesia for a few weeks now, mostly for prototyping voice interactions for a SaaS dashboard. The "intent" feature is the part that keeps blowing my mind—it feels like magic when it correctly routes a user's rambling sentence into a specific action.
But as someone who lives in Figma variables and prototyping logic, the "how" is killing me. I get the basic sales pitch: you feed it an utterance, and it matches to a pre-defined intent. Easy. But the black box is frustrating.
Can someone break down the actual mechanics? Like, when I set up an intent called "cancel_subscription" with a few example phrases, what's happening when a user says "Yeah I need to stop my plan, please"? Is it just a fancy keyword matcher with synonyms? Or is there some kind of lightweight local model doing real-time parsing? The documentation is a bit vague on the technical specifics.
I'm especially curious about the confidence scoring. Is that based on semantic similarity to the training phrases, or is it looking at the entire context of the conversation state? Getting this right is huge for building reliable voice UX, and I don't want to design flows based on a misunderstanding of the underlying tech. Anyone from the team or power users who've dug into this?
It's a great question, and that black-box feeling is something a lot of us grapple with when trying to build reliable systems. From what I've seen in practice and pieced together from their engineering talks, it's likely a blend of both approaches you mentioned.
It starts with the embeddings from their voice model - your example phrases and the user's utterance get converted into numerical vectors capturing semantic meaning, not just keywords. So "stop my plan" and "cancel subscription" are close together in that vector space. The confidence score probably reflects that cosine similarity distance. But it's rarely just that static comparison; there's usually a thin classification layer on top trained on those examples to make the final call, which helps with phrasing it hasn't seen before.
The real trick for routing often comes from that confidence threshold you can set. A low confidence might trigger a clarification question, which is where you'd design a fallback flow. It's less about analyzing a full conversational context and more about the semantic match quality of that single utterance against your defined set. For prototyping, I'd suggest stress-testing it with a wide range of paraphrases to see where that threshold needs to be before you lock in a design.
—HR
That vector similarity point is spot on. From my testing, the few example phrases you give act like anchors in that semantic space. "Cancel my subscription" and "I need to stop my plan" are clustered close together.
The magic sauce for me is how it handles completely unseen phrasing. I threw "Just end it all" at a test intent for cancellation, and it matched with high confidence. That's the thin classifier layer at work, going beyond simple synonym matching.
For UX flows, treat the confidence score as "semantic distance from your examples." Low scores mean the user's phrasing drifted far from your anchor points, so maybe trigger a clarifying prompt. Works like a charm.
Let's build better workflows.