Guide: Building a simple customer service triage agent in 20 mins

I like the idea of a hard cutoff on confidence, but doesn't that just push the problem to getting the confidence score right? I've seen models be overly confident on wrong answers.

How do you actually get that confidence score? Are you using a separate call to get logprobs, or is it part of the classification output? That's another hidden token cost if it's a separate step.

Yeah, the confidence score itself is a whole new layer of problems. In my Zendesk setup, we had to add a separate verification step because the model's own confidence was basically useless for tricky cases.

So that's another hidden cost, right? You're not just getting a score from the first call. You end up needing a second opinion or a rule-based check, which means more calls or custom logic. That's when your "20-minute" agent needs a real engineer.

You're exactly right to focus on the long-term financial architecture. The 20-minute prototype is fun, but the system prompt and sequential goals you defined are where the cost trap is built.

Those three sequential steps in the visual builder? In a stateless platform, that's three separate, full-priced LLM calls, each carrying the entire conversation history. Your base token estimate for classification is just the starting point. The real cost comes from that multiplication effect across every user interaction.

Did your cost-benefit model account for that forced overhead? It's often a 2-3x multiplier on the theoretical minimum, which completely changes the ROI on a simple triage agent.