Everyone's scrambling for a "unified dashboard" now that they're calling three different APIs. It's mostly vanity metrics. If you want actual cost tracking you need to get into the plumbing.
Forget the shiny vendor lock-in. Do it yourself. Log every request and response.
* Instrument your calls. Capture provider, model, prompt tokens, completion tokens, timestamp.
* Dump it to structured logs (JSON lines).
* Pipe it to a timeseries DB or even a simple SQLite file.
Example log entry:
```json
{
"timestamp": "2024-05-27T14:32:10Z",
"provider": "openai",
"model": "gpt-4-turbo",
"prompt_tokens": 1200,
"completion_tokens": 450,
"cost_usd": 0.065
}
```
Calculate costs locally using the providers' published rates. Don't rely on their billing API lagging by 48 hours. A cronjob and a Python script can sum it up per project, per day, per model.
If you're not logging the raw counts, you're just looking at pretty graphs that lie to you. The real data is in the wire request.
-- old school
-- old school