Hey everyone, new here 👋
I've been experimenting with LangChain for a few weeks and keep hearing about LangSmith for monitoring. The hosted version looks great, but for my company's data, self-hosting would be better.
Has anyone actually set up the self-hosted version? I'm a bit worried about the ongoing maintenance. Is it a lot of work to keep it running smoothly? Like, do you need a dedicated devops person, or can a small team handle it? Would love to hear about real experiences!
Yes, I've been running the self-hosted LangSmith stack in production for about six months now. The maintenance burden isn't trivial, but your main problem won't be keeping the services alive. It's everything else that orbits it.
The deployment itself, using their Docker Compose or Helm charts, is straightforward. The containers run. The real work starts with:
- Scaling the storage backend (Postgres/Opensearch) as trace data balloons.
- Managing data retention policies yourself, which means writing cleanup jobs.
- Handling upgrades, which have broken our dashboards twice because of schema changes in the monitoring tables.
- You also become responsible for securing the whole thing, which adds audit and patching cycles.
A small team with decent DevOps awareness can handle it, but you are effectively taking on a new data pipeline product. You need to monitor its health, its resource usage, and its data growth. If you don't have someone who can write a cron job to prune old runs or debug a borked OpenSearch cluster, you'll have a bad time.
Honestly, the cost-benefit only makes sense if you have stringent data sovereignty requirements that rule out their cloud. Otherwise, the operational overhead will quickly eclipse your actual LLM development work.
—davidr