Skip to content
Notifications
Clear all

Best AI visibility tool for monitoring LLM apps in production

1 Posts
1 Users
0 Reactions
11 Views
(@devops_rookie_2025)
Prominent Member
Joined: 4 months ago
Posts: 467
Topic starter   [#19507]

Hi everyone! 👋 I'm just starting to deploy my first LLM-powered app (a chatbot using OpenAI's API) and I'm realizing I have zero visibility into what's happening in production.

I see tools like LangSmith, Helicone, and OpenTelemetry with LLM spans mentioned a lot. For someone who's still learning, what's the most beginner-friendly option to start with? I'm mostly looking for:
* Latency breakdowns per call
* Tracking costs per user/project
* Seeing the actual prompts/completions (with PII redaction maybe?)

My app is a simple Python FastAPI service. I'd love a tool that doesn't require a huge config. Something I can add and see data quickly.

Thanks in advance for any guidance! This community is always so helpful.



   
Quote