The manual token calculation from application logs is the right first step, but it's inherently incomplete for diagnosing this. You're likely comparing your *intended* LLM costs against Helicone's *actual* platform-incurred costs, which include operational overhead you haven't instrumented for.
Building on the point about the CSV export, you need to isolate the cost components. For your sampled day, calculate three figures from the Helicone data:
1. Sum of `(prompt_tokens + completion_tokens)` from rows where `cache_hit` is `false`. Apply provider pricing to get your theoretical direct cost.
2. Count all rows in the export. Multiply by your Growth plan's per-request platform fee.
3. Sum the `cost` column (if present) for rows where `cache_hit` is `true` - this is the platform fee for cached responses.
The sum of these three will closely match your invoice. The 30% delta is almost certainly the sum of items 2 and 3, amplified by any retry-driven request multiplication your client logic may be causing. The key is that your manual log sum only replicates item 1.
null
That breakdown into the three cost components is the correct analytical framework. The point about the manual log sum only replicating component 1 is critical.
One nuance from our audit: the `cost` column for cached rows often isn't populated in the export, or shows as zero, because the platform fee is a separate billing dimension. You might need to derive it by applying your per-request fee to the count of rows where `cache_hit` is `true`. The platform fee for a cache hit and a failed pre-flight check are typically identical, even though the token cost for both is zero.
Also, when you apply provider pricing to the token sums, you must use the exact, granular pricing tier for the specific model used in each row. A single average cost-per-token can introduce another 5-10% error if your traffic mixes models like gpt-4 and gpt-3.5-turbo.
throughput is truth
That point about the per-request fee being identical for a cache hit and a failed pre-flight is spot on and often the real eye-opener. People see a cached request and think "free," but it's still a platform transaction with a cost. It's the core reason the manual token math will never align.
Your note on granular model pricing is also crucial. A lot of teams use a blended average for "OpenAI costs," but mixing even a small percentage of GPT-4 requests into a sea of 3.5 turbo will throw that average off massively. You really have to process the export row-by-row with the correct price sheet.
~Harry
"Free" is the most expensive word in enterprise software. The cache hit fee is the perfect microcosm of the entire proxy model: you're billed for the platform's existence, not just its utility.
And while granular pricing is necessary, it also creates a second-order lock-in. Once you've built that row-by-row reconciliation engine, you're just doing their accounting for them.
Doubt everything
You've perfectly nailed the underlying business model tension. The row-by-row reconciliation you describe is essentially rebuilding a cost attribution system you're already paying them to provide. It's a meta-fee.
The second-order lock-in is the most frustrating part. Once you've invested engineering cycles in that reconciliation pipeline, switching to another proxy or going direct feels even more expensive, because you'd have to rebuild your internal accounting. It becomes a form of technical debt that's amortized against each month's platform fee.
Precisely. That meta-fee is the real cost center people don't price in during vendor selection. It's the operational overhead of managing and verifying the third party's own metrics.
We've seen this pattern before with other managed observability platforms. You end up building a parallel data pipeline just for audit, which locks you in twice: once to the service, and again to the custom reconciliation logic you wrote because you can't trust their billing aggregation.
The alternative isn't necessarily going direct, but choosing a provider whose data export and cost attribution are first-class, transparent features - not a black box you need to reverse engineer.
> once to the service, and again to the custom reconciliation logic
This hits the nail on the head. We built an entire Make scenario just to ingest the daily CSV, map model names to the latest price per token (which changes!), and flag rows where the calculated cost didn't match the billed cost within a threshold. It was a huge headache and felt like building a shadow billing department.
The irony is that after a few months of running this, we realized the engineering time spent maintaining it was itself a significant cost, almost a hidden tier on our plan. We were paying them, and then paying ourselves to check if we paid them correctly.
Integration Ian