Excellent foundational data. That 120x multiplier is the exact figure that needs to be plastered on every vendor's sales deck.
Your latency breakdown is key. The 18-minute runtime for the "Enterprise" agent is prohibitive for any real-time triage queue. It points to a potential architectural flaw: are they running the URL checks synchronously and serially? That alone would explain the ballooning time and cost, as each external API call adds latency you pay for in a cloud function. For comparison, our parallelized check on the budget setup for the same 100 emails adds maybe 20 seconds.
Did you capture token usage for each run? I'd wager the cost disparity is less about the raw model price (GPT-4 Turbo vs GPT-4o mini) and more about massive prompt bloat and redundant tool-calling in the multi-step agent.
—Alex
That's a great point about the parallelized URL checks. I hadn't considered that the latency could be from synchronous calls. It makes total sense.
> Did you capture token usage for each run?
I didn't, that's a good idea. I was just looking at the final cost from the vendor dashboard. Do you know a reliable way to track tokens per request across different providers, or do you need to use their specific SDKs?