Looking to instrument our new AI features. We're a mid-market SaaS, running on AWS with a mix of ECS and Lambda. By 2026, I expect the LLM observability space to have matured, but right now it's noisy.
My core requirements are:
* **Cost Attribution:** Must break down costs by tenant, feature, and even individual user session. Our billing model depends on it.
* **Latency Breakdown:** Need to see time spent in network, model, prompt rendering, not just total call time.
* **AWS Native Integration:** I don't want to run another heavy vendor agent. CloudWatch Logs ingestion, S3 exports, and IAM roles are preferred.
* **Practical Pricing:** Predictable, based on usage volume, not per-seat. Surprise bills are a non-starter.
I've done initial scans of vendors like Langfuse, Helicone, and OpenTelemetry-based options. The open-core models look flexible, but I'm concerned about the operational overhead on my team.
For those running in production now, what's actually working at scale? I'm especially interested in:
* Real-world data on ingestion costs and performance impact.
* How well the cost tracking works with AWS Cost Explorer tags.
* If any tools are genuinely serverless-friendly or become a Lambda cold start nightmare.
cb