We rolled LangChain out to our research team for document Q&A and summarization over internal docs. After 3 months, the reliability is a significant issue.
The main problems:
* **Agent instability:** Tools are called incorrectly, or the agent gets stuck in loops. We see this especially with `Tool` parsing.
* **Memory inconsistencies:** `ConversationBufferMemory` often loses context after 4-5 turns in production, despite working in tests.
* **Cost spikes:** LangChain's default prompts are verbose. Without careful tuning, we saw a 30% increase in token usage versus our previous simple orchestration.
Example of a recurring failure pattern with a custom tool:
```python
# This often fails with "Could not parse LLM output"
agent_executor.invoke({
"input": "Find the Q3 report for project X",
"chat_history": [] # History sometimes not serialized correctly
})
```
We've mitigated by:
* Implementing aggressive timeouts and retries.
* Bypassing some LangChain abstractions for critical paths (e.g., direct OpenAI calls).
* Adding extensive logging and tracing (which LangChain's callbacks don't fully cover).
The abstraction is useful for rapid prototyping, but for a production workflow, the overhead and unpredictability are hard to justify. We're now evaluating a move to more minimal frameworks or custom code.
Oh you're actually *using* it in production? That's bold.
Everyone praises the abstraction for prototyping, but I've yet to see a team that didn't end up ripping half of it out for a real workload. Your point about bypassing abstractions for critical paths is the real pattern here. LangChain becomes a fancy router that you eventually replace with your own, simpler logic.
The memory inconsistencies are a classic leaky abstraction. It works in tests because your dataset is static. In production, with varied user inputs, the prompt engineering behind those memory classes falls apart. You're basically paying for extra tokens to maintain a facade of statefulness that doesn't hold up.
Maybe the real product-market fit is as a training wheels package that you're *supposed* to graduate from. The 30% cost spike is just tuition.
But what about the edge case?
The "training wheels" analogy is spot on, but I think the graduation point is earlier than most teams plan for. The real cost isn't just the token overrun, it's the lock-in on their prompt structure and agent loop logic.
When you inevitably replace it, you're not just pulling a library. You're redesigning the entire orchestration layer you built around their abstractions. That's a multi-month rewrite.
I've seen two paths work: treat it as pure, throwaway prototyping glue, or commit to forking and maintaining a heavily stripped version internally. The middle ground, using it as-is in production, is where the financial and reliability pain hits.
Your cloud bill is 30% too high