Hi everyone! I'm just starting to explore LangChain for a small internal tool at my company. We're building a simple chatbot to help our support team find answers faster.
I'm looking at LangSmith for monitoring, but our volume is super low—definitely under 10k calls per day to start. LangSmith's pricing seems more geared towards bigger teams? 😅
What are you all using to keep an eye on your smaller LangChain apps? I really just need to see if prompts are failing, track token usage, and maybe log a few traces. Something simple and cost-effective for this stage would be perfect.
I run a similar small internal chatbot using LangChain for a 50-person SaaS team. We handle around 5k-7k calls/day and went through this evaluation two months ago.
**Fit / Target User**: LangSmith is built for LLM devs, but is heavy for pure monitoring. It's fantastic for prompt engineering and debugging chains, which you may not need if your app is stable.
**Real Pricing**: LangSmith starts at $25/user/month on a team plan. For under 10k calls, you're looking at ~$50/month minimum. The hidden cost is its complexity; you pay for dev features you might not use.
**Deployment Effort**: OpenTelemetry (OTel) with a Grafana stack takes about 2-3 days to wire up initially. You add a few lines to instrument LangChain, point it to your OTLP endpoint, and build dashboards.
**Where it Clearly Wins**: For pure cost, OTel to a self-managed Grafana/Prometheus/Loki stack wins. It's nearly free at your volume if you have existing infra. You get full control over retention and alerts.
I'd pick OpenTelemetry with your existing Grafana setup. It's the most cost-effective for just monitoring failures and token usage. If you were actively developing and tuning prompts weekly, I'd lean LangSmith. Tell us if you have a Grafana instance already running and how much dev time you can spend on setup.
Totally feel you on the pricing. For a simple, stable bot, I actually just used LangChain's built-in callback handlers to ship logs to Datadog for months. It was a bit hacky, but it cost me nothing extra since we already had Datadog.
If you don't have an existing monitoring stack, a quick win is setting up a simple dashboard with the OpenAI SDK directly. You can log token usage and errors from their responses without needing a full tracing system. It's not perfect, but for under 10k calls, it gets the job done.
What's your current infra look like? Any logging tool already in place?
It's not marketing, it's logic.