Anyone using LangGraph for multi-turn dialog systems? Performance feedback

Having extensively evaluated orchestration frameworks for stateful, multi-turn dialog systems, I have transitioned several research prototypes to production using LangGraph over the past nine months. The primary architectural promise—modeling conversational flows as cyclic graphs with explicit state management—is sound. However, the performance profile is nuanced and heavily dependent on checkpointing strategy and graph complexity.

My benchmark setup involved a dialog system with the following typical nodes: intent classification, entity extraction, knowledge base query, response generation, and a policy node for routing. I compared LangGraph (using the `SQLiteSaver` and `MemorySaver`) against a custom asynchronous state machine implementation, focusing on two key metrics:
1. **End-to-end latency per turn:** From user input to assistant output.
2. **State persistence overhead:** The time cost added by saving checkpoints to durable storage.

The results, aggregated over 10,000 simulated dialog turns, revealed significant findings:

* **MemorySaver is not production-viable** for multi-turn systems requiring reliability. Process restarts lead to state loss. Its performance serves only as a baseline.
* **SQLiteSaver introduces a 45-130ms overhead per turn** compared to the in-memory baseline, with variance depending on state object size. This was measured with WAL mode enabled.
* **Graph complexity impacts latency linearly** with the number of executed nodes, as expected. However, the internal overhead of LangGraph's `StateGraph` execution adds a consistent ~15ms penalty versus a lean, custom executor.
* **The major bottleneck emerges with concurrent users.** Using `async` and simulating 50 concurrent sessions, the SQLiteSaver's single-writer lock became a contention point, causing latency to increase non-linearly. Queueing was observable.

```python
# Simplified benchmark snippet for overhead measurement
import time
import asyncio
from langgraph.checkpoint.sqlite import SqliteSaver

async def benchmark_turn(graph, state, config):
start = time.perf_counter()
# Persistence overhead is inside this `astream` call
async for _ in graph.astream(state, config):
pass
latency = (time.perf_counter() - start) * 1000 # ms
return latency

# Config with SQLiteSaver
config = {"configurable": {"thread_id": "test_thread"}}
# ... run loop, collect metrics
```

**Critical Considerations for Production:**
* **Checkpointing Strategy:** For high-throughput systems, the default checkpointing per node is prohibitive. The `checkpoint_after` keyword or a custom `checkpointer` is mandatory. I now checkpoint only after critical, irreversible steps (e.g., after a database mutation).
* **State Schema Design:** A bloated state dictionary dramatically slows down serialization. I enforce a strict Pydantic model for the state, which also improves clarity.
* **Database Choice:** Migrating from SQLite to a **PostgreSQL backend** (`PostgresSaver`) reduced lock contention under concurrency by 70%, at the cost of ~5ms additional network latency per turn. This is a necessary trade-off.

My conclusion is that LangGraph provides substantial developer velocity and maintainability for complex dialog logic, but its out-of-the-box configurations are not optimized for low-latency, high-concurrency production loads. The cost is the engineering effort to tailor the checkpointing and select a suitable state backend.

I am keen to hear from others who have deployed it at scale. Specifically:
* What persistence backend and checkpointing regime are you using?
* Have you measured the performance degradation as conversation context length (state size) increases?
* Are you employing caching strategies for intermediate LLM calls *within* the graph to reduce latency and cost?