Everyone's raving about Langfuse's built-in UI for comparing LLM traces. Fine. But their real power is the API. You can roll your own comparison dashboard in an afternoon and avoid their opinionated layout.
Here's the gist: fetch your traces, pick your comparison logic, and build a simple table. No magic, just HTTP calls and a bit of scripting.
Example: pulling a generation and its scores for a side-by-side view.
```bash
# Get a specific trace with its observations
curl -s -X GET "https://cloud.langfuse.com/api/public/traces/YOUR_TRACE_ID"
-H "Authorization: Bearer $LANGFUSE_SECRET_KEY"
-H "Content-Type: application application/json" | jq '.observations[] | select(.type=="GENERATION")'
```
Then you can pull another trace by `name` or `session_id` and compare the `output` and any `scores`. Dump it into a simple HTML table or even a CLI diff.
Key points:
* You control the comparison dimensions (latency, cost, score, output length).
* You can embed it in your internal tools.
* You're not stuck waiting for them to add the specific comparison view you need.
Downside? You have to build it. But if you've ever configured a Jenkins pipeline, this is trivial.
-- old school
-- old school
Completely agree about the API being the real value. I've built a few of these internal dashboards, and the ability to compare traces across different migration runs, timestamps, and model versions is where it shines. Your curl example is a good start, but the real power is in pagination and bulk fetch when you're comparing dozens of traces from a single session.
One caveat: you'll want to cache those trace objects locally, maybe in a small SQLite instance, if you're building a tool for regular use. Hitting the API for every render gets slow. I usually schedule a nightly job to pull the last 24 hours of traces for our key pipelines and store the observations as JSONB. Then the UI is just querying that local store.
Also, don't forget the `/api/public/scores` endpoint if you're doing comparison across custom evaluations. You can pull all scores for a set of trace IDs in one go and map them to your dimensions.