We just finished a major rollout of LangGraph for a multi-agent orchestration layer, moving from 5 prototype agents to 50 in production. The promise of built-in persistence, human-in-the-loop, and cyclic workflows was a huge draw, but scaling hit us with some unexpected friction. I thought I'd share our breakdown of what actually broke and the (sometimes janky) fixes we implemented.
The main pain points weren't with the graph logic itself, but with the surrounding ops and resource management:
* **Memory Explosion:** The default in-memory checkpointing was a non-starter. At scale, with concurrent runs, our memory usage ballooned. We switched to the Redis checkpoint persister, but then faced serialization issues with custom objects. The fix was a mix of using Pydantic for all state models and implementing a custom encoder for the Redis store.
* **Concurrency Bottlenecks:** The `StateGraph` with parallel edges is great, but we initially had agents blocking each other on shared resources (like a vector DB connection pool). We had to implement a semaphore system *outside* of LangGraph, using FastAPI's limits, to throttle certain nodes. Not ideal, but it worked.
* **Cost Surprise:** This is my wheelhouse. We're on OpenAI for most LLM calls. LangGraph's built-in tracing to LangSmith is fantastic for debugging, but we didn't realize the volume of metadata and traces we'd generate. Our LangSmith bill spiked more than expected. We dialed down the sampling rate for successful runs and only do full tracing for error paths now.
* **"Checkpoint Spam":** With 50 agents, the number of saved checkpoints (even with pruning) in Redis was enormous. We wrote a nightly cleanup job that purges checkpoints older than 7 days for completed graphs, unless they were flagged for audit.
The biggest lesson was that LangGraph moves the complexity from "orchestrating the DAG" to "managing the state and concurrency." You get a robust framework for the workflow, but you own the infrastructure for its persistence and scaling.
Has anyone else hit scaling walls with the checkpointers? We're looking at the Postgres option next, but I'm wary of adding another DB just for state. Would love to compare notes on the operational overhead vs. the in-memory prototype phase.