Skip to content
Notifications
Clear all

Check out what I made: a custom cost report using Helicone API.

4 Posts
4 Users
0 Reactions
2 Views
(@finops_tracker_99)
Estimable Member
Joined: 5 months ago
Posts: 87
Topic starter   [#16622]

I've been using Helicone for a few months to track our OpenAI costs, and while the dashboard is great for a high-level view, I needed something more granular for our weekly FinOps sync. The built-in reports didn't quite slice the data the way I wanted—I needed to attribute costs per internal project and developer, across specific model families.

So, I built a script that pulls data via the Helicone API and generates a custom CSV report. It's been a game-changer for spotting waste, like catching a stray `gpt-4-32k` call in a project that should only be using `gpt-3.5-turbo`.

Here's the core of the Python script. It fetches request logs, aggregates them, and spits out a weekly summary.

```python
import requests
import pandas as pd
from datetime import datetime, timedelta

HELICONE_API_KEY = "your_key_here"
url = "https://api.helicone.ai/v1/request"

headers = {
"Authorization": f"Bearer {HELICONE_API_KEY}",
}

# Get last week's data
end_date = datetime.utcnow().date()
start_date = end_date - timedelta(days=7)

params = {
"from": start_date.isoformat(),
"to": end_date.isoformat(),
"limit": 1000 # Adjust as needed
}

response = requests.get(url, headers=headers, params=params)
data = response.json()

# Process and aggregate
df = pd.DataFrame(data)
# Add custom logic here to map 'userId' or 'customProperty' to internal projects
df['cost'] = df['totalCost'] # Use actual cost field from Helicone
df['model_family'] = df['model'].apply(lambda x: x.split('-')[0]) # Simple grouping

summary = df.groupby(['project_tag', 'model_family', 'userId']).agg({
'cost': 'sum',
'id': 'count' # Request count
}).rename(columns={'id': 'request_count'})

summary.to_csv('helicone_weekly_report.csv')
```

Key takeaways from running this:
* It exposed a pattern of low-utilization `gpt-4` calls during off-hours—turns out a scheduled job hadn't been downgraded.
* By tagging requests with a `project_tag` custom property, we finally got accurate cross-team chargebacks.
* We now combine this data with our AWS/Azure bills for a unified cloud cost view.

The API is straightforward. The main challenge was mapping Helicone's `userId` and custom properties to our internal directories. Once that was done, the aggregation was simple.

Has anyone else built custom reports? I'm particularly interested if you've found a way to correlate this LLM cost data with compute costs from your cloud provider for a total "cost per AI feature" metric.



   
Quote
(@data_shipper_joe)
Reputable Member
Joined: 2 months ago
Posts: 184
 

Nice! I've been meaning to poke at the Helicone API for a similar reason - their dashboard is slick but sometimes you just need your own aggregates. Your point about catching the stray `gpt-4-32k` call is spot on; that's exactly the kind of thing that slips through the cracks.

One thing I'd watch out for with that `limit=1000` param is pagination. If your volume grows, you might start missing data. The response usually includes a cursor for the next page. I learned that the hard way with another API a while back!

Have you thought about pushing these weekly aggregates back to your data warehouse? I started sending mine to BigQuery so the finance team can join it with other cloud spend. Feels like it completes the loop.


ship it


   
ReplyQuote
(@emmaj)
Estimable Member
Joined: 1 week ago
Posts: 92
 

Great catch on the pagination! That totally bit me a few weeks in. I had to add a loop to follow the `cursor` in the response until it's `null`. It's a lifesaver now that our volume is picking up.

Sending the aggregates to BigQuery is such a solid idea, it really does close the loop. We're a Snowflake shop, so I've been piping the weekly CSV into a stage and then loading a table. The finance team loves it because they can finally join our AI spend with AWS and Datadog costs on a single dashboard. Have you set up alerts on your BigQuery data for cost anomalies?



   
ReplyQuote
(@emilykim)
Estimable Member
Joined: 1 week ago
Posts: 75
 

That's a clean approach for weekly aggregation. We used a similar script but ran into issues with the default cost attribution. The API's `cost_usd` field sometimes didn't match our internal calculations when we had custom pricing agreements.

We ended up adding a secondary check using the raw token counts and the latest pricing sheet from OpenAI's website. It added an extra step, but the reconciliation saved us a few percentage points on our monthly bill. Did you validate the reported costs against another source initially?


Your bill is too high.


   
ReplyQuote