Just built a Grafana plugin to visualize Traceloop cost data. Sharing the code.

I've been running Traceloop in production for tracing our LLM pipelines for about three months now. The cost data it collects is invaluable—seeing token counts, provider costs, and latency per span—but the built-in UI is more of a trace inspector than a dashboard for trend analysis. I needed a way to see cost-per-model trends over time and correlate spikes with deployments or traffic changes.

So I built a simple Grafana plugin to pull cost metrics directly from Traceloop's OpenTelemetry data and visualize them. It's essentially a data source plugin that queries Traceloop's collected metrics, which are stored in whatever backend you're using (I'm using Prometheus). The key was mapping Traceloop's span attributes to meaningful time-series metrics.

Here's the core logic that transforms Traceloop spans into aggregate cost metrics. This runs in a small service that exposes a `/metrics` endpoint for Prometheus to scrape.

```python
# tlp_cost_exporter.py
from opentelemetry import trace
from prometheus_client import Counter, Gauge, start_http_server
import json

# Prometheus metrics
total_inference_cost = Counter('traceloop_inference_cost_total',
'Total inference cost in USD',
['model', 'provider'])
avg_token_latency = Gauge('traceloop_avg_token_latency_ms',
'Average latency per token in milliseconds',
['model', 'provider'])

def process_span(span_data):
"""Extract cost and latency from Traceloop span attributes."""
attrs = span_data.get('attributes', {})
model = attrs.get('gen_ai.model', 'unknown')
provider = attrs.get('gen_ai.system', 'unknown')
input_tokens = int(attrs.get('gen_ai.token_count.prompt', 0))
output_tokens = int(attrs.get('gen_ai.token_count.completion', 0))
cost = float(attrs.get('gen_ai.gen_ai.request.cost', 0))
duration_ms = span_data.get('duration_ms', 0)

if cost > 0:
total_inference_cost.labels(model=model, provider=provider).inc(cost)

total_tokens = input_tokens + output_tokens
if total_tokens > 0 and duration_ms > 0:
avg_token_latency.labels(model=model, provider=provider).set(duration_ms / total_tokens)
```

The Grafana plugin configuration (JSON) then queries these exposed metrics. I set up panels for:
* **Total daily cost by model** (stacked bar chart)
* **Cost per 1k output tokens** (trend line, per model)
* **Latency per token correlation with cost** (scatter plot, using queries joined on model/provider labels)
* **Top 5 most expensive traces in the last 24h** (table view, requires querying trace data directly)

The biggest hurdle was normalizing the cost data, as Traceloop records the raw cost from the provider API call, but you need to be careful with units (some providers return cost in cents, others in dollars). I added a normalization map in the exporter.

This setup now gives me a real-time view of which models are driving our inference budget and whether performance regressions are increasing cost/token. For example, we spotted a 15% cost increase last week that correlated exactly with a switch to a newer model version that had higher latency-per-token, nullifying its slightly better accuracy. Without this dashboard, we'd have only seen the higher bill at the end of the month.

The code is rough but functional. If anyone wants the full Grafana plugin source and the exporter, I can post the repo link. I'm particularly interested if others have built similar integrations and how you're handling dimensionality—adding labels for team, project, or user ID can make the cost attribution much more powerful.