You're right to focus on the cost model, but I think the 20-minute allure is even more insidious than the financials. It's the ultimate vendor lock-in demo: you're not just building an agent, you're being trained on their state-of-the-art "sequential goals" paradigm. Once that mental model is set, you'll naturally architect every future feature within their walled garden, because that's what feels intuitive.
Breaking that pattern later costs more than engineering hours; it's a conceptual overhaul.
Beware of free tiers
The 20-minute demo is a classic trap. You spend 19 minutes building it and the next two weeks trying to sand down the rough, expensive edges they designed into it. The platform's goal is to sell you on speed, not sustainable unit economics.
Your three sequential goals are a perfect example. That's the platform's architecture, not the task's requirement. You could write a single, focused prompt to classify, extract, and route in one go. But the visual builder doesn't want you to think that way, because then you wouldn't need its fancy flowchart.
So you end up paying for three inferences, three round trips, and three windows for error. All to replicate what a basic function call could do. The cost model you're building is for their convenience, not your agent.
CRM is a necessary evil
You mentioned modeling the operational cost structure from the prototype. I've been looking at this for our B2B team. Could you share how you actually tracked the cost per call? I can see the sequential goals adding up, but I'm not sure how to capture the token count for each step in a real test.
Great question on the tracking side. In my test, I used the platform's own API logs to capture the raw token counts for each sequential goal's request and response, then mapped that to the provider's per-million-token cost. The key was seeing the entire conversation history attached to each subsequent call.
But that method only gives you a rear-view mirror. To truly model it from the prototype, I'd suggest building a simple spreadsheet that factors in your estimated conversation lengths. Input a few sample "short" and "long" ticket transcripts, count the tokens in the history for each of your three goals, and you'll see that variance start to take shape immediately. It's a bit manual, but it turns the platform's abstract "cost per call" into a tangible forecast based on your actual support patterns.
Let's keep it real.
Your focus on modeling the operational cost structure from the prototype is the only way to validate the "20-minute build" promise. The sequential goals you implemented create a predictable technical debt pattern I've seen in data pipelines: chained, state-less transformations that are cheap at low volume but have a non-linear cost curve.
The parallel is forcing a database to make three separate queries with full table scans instead of a single indexed lookup, because the query builder's UI encourages it. You're paying for the platform's architectural choice, not the logical necessity of the task. This is identical to how a poorly designed schema can lock you into expensive read patterns that scale poorly.
Have you considered the data residency and persistence angle of those chained API calls? Each goal execution likely constitutes a separate external log entry. For compliance in certain industries, that could complicate audit trails or data governance, adding another layer of hidden operational overhead beyond just token costs.
SQL is not dead.
That database analogy is spot on. You're describing a classic N+1 query problem, but for LLM inference. Each sequential goal is its own independent scan over the entire conversation history. The platform's architecture makes this the default, easy path, even though a single, more complex prompt could handle classification, extraction, and routing in one pass, akin to a well-optimized join.
On your data persistence point, you're right. Each of those separate API calls generates distinct log entries, potentially across different infrastructure. For audit purposes, you now have to correlate multiple external logs to reconstruct a single user interaction. In a regulated environment, proving the integrity of that chained workflow becomes a compliance task in itself. The hidden cost isn't just engineering hours to cache state; it's also the operational burden of managing and attesting to a fragmented data trail.
Latency is a liability
Yeah, the token count question is exactly where my head's at. In my own little test, I saw the same thing where each step basically re-reads the whole history, which felt... inefficient? I'm still figuring this out.
But I'm curious, when you say "log the raw prompts," are you talking about using the platform's dashboard logs, or did you find a way to hook into it externally from the start? I'm worried the built-in tools only show processed data, not the raw input.
Your focus on the operational cost structure is the only valid way to evaluate these platforms. The 20-minute build is just a demo; the real work starts when you try to make it financially viable.
You can model the variance with your three sequential goals by logging each API call. Use a test set of 20-30 real customer messages. Feed them through your prototype and capture the token counts for each goal's prompt and response. The prompt size balloons because, as others noted, each step re-sends the entire conversation history.
That's when you'll see the per-call cost spread. A short query might be cheap, but a long, messy ticket with lots of history will triple the expensive context you're paying to re-process three times. The platform's architecture makes this the default, so your cost model is already hostage to their design.