Everyone's raving about AI observability platforms like they're the second coming. So we tried Traceloop for a few months on a new project. Just rolled it back to some custom Python logging and a dashboard. The complexity tax wasn't worth it.
Our use case was tracking LLM calls for a handful of internal tools. We're talking maybe a few hundred calls a day, single region, one primary model provider. Traceloop gives you the whole kitchen sink: traces, spans, evaluations, the works. For a complex microservice mesh doing RAG with ten different models, I'm sure it's a godsend. For us, it was like using a particle accelerator to crack an egg.
The main friction was the sheer volume of *stuff* we didn't need. The UI is built for deep, interconnected traces. Ours were trivial chains. The pricing model scales with spans, so even our simple workflow started to feel like a cost center for data we weren't analyzing. The final straw was debugging a latency issue. Flipping through their detailed trace view was overkill; we just needed timestamps and model names. We wrote a 50-line decorator that logs to CloudWatch and a tiny Grafana panel in an afternoon. Problem solved, cost zero.
I see the appeal for the right scale and complexity. But there's a real survivorship bias in these reviews. The happy users are the ones with problems complex enough to justify the platform. If your workflow is simple—single LLM, few steps, low volume—you're probably just paying for a fancy wrapper around `print()`. Sometimes the manual way isn't technical debt; it's just the right tool.
Anecdotes aren't data.