I've been evaluating LangGraph for a production orchestration layer, specifically for a complex, multi-agent ETL pipeline that handles real-time data enrichment. The built-in LangSmith integration is fine for development, but it's not a viable solution for operations at scale where we need to correlate LangGraph events with our broader application metrics, infrastructure logs, and existing alerting systems. LangSmith is a silo, and our ops team lives in Datadog.
The core question is straightforward: has anyone moved beyond the provided callbacks and actually streamed LangGraph execution traces, step durations, node inputs/outputs, and—critically—errors to a third-party APM like Datadog? I'm not talking about just logging a "run ended" event; I need granular, structured data that can be queried, alerted on, and visualized alongside our Kafka lag, database load, and cloud function invocations.
I've dissected the `LangGraphTracer` and callback handlers. The naive approach would be to write a custom handler that pushes to Datadog's HTTP API, but the volume and cardinality of trace data are concerns. I'm looking for proven patterns.
Key specifics I need to understand:
* **Instrumentation Depth:** Did you instrument at the `Graph` level, the `Node` level, or both? Capturing the high-level graph execution is useless without the node-level granularity for debugging bottlenecks.
* **Data Schema:** What did you actually send? A flattened JSON structure of the `RunState`? Did you transform it or sample it to avoid cost overruns?
* **Error Handling:** How did you ensure that logging failures themselves don't crash the graph execution? Async fire-and-forget? A durable queue?
* **Performance Impact:** What was the observable latency overhead? Our graphs must complete in sub-second timeframes.
A skeleton of my current, unsatisfactory prototype is below. It logs, but it's brittle and doesn't handle edge cases or sampling.
```python
from langgraph.callbacks import BaseCallbackHandler
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.logs_api import LogsApi
from datadog_api_client.v2.model.http_log_item import HTTPLogItem
import json
import threading
class DatadogLoggingHandler(BaseCallbackHandler):
def __init__(self, service_name="langgraph"):
self.config = Configuration()
self.service = service_name
self._lock = threading.Lock()
def on_chain_start(self, serialized, inputs, run_id, **kwargs):
# This feels inadequate
log_item = HTTPLogItem(
message=f"Graph chain started: {run_id}",
service=self.service,
ddtags=f"run_id:{run_id},event:chain_start",
additionalProperties={
"run_id": run_id,
"inputs": str(inputs)
}
)
self._send_log_async(log_item)
def _send_log_async(self, log_item):
# This is the problematic part - needs proper async, batching, and error isolation
def _send():
with ApiClient(self.config) as api_client:
api_instance = LogsApi(api_client)
try:
api_instance.submit_log([log_item])
except Exception as e:
print(f"Datadog submission failed: {e}") # Cannot print in production
thread = threading.Thread(target=_send)
thread.start()
```
This approach is insufficient for production. The logging is too shallow, the async mechanism is naive, and there's no cost control. I suspect the correct answer involves a background thread with a batching buffer and a separate error queue, but I'd rather not reinvent the wheel if someone has already solved this.
Before I commit to building a dedicated sidecar service to ingest and forward LangGraph traces, I want to know if there's a cleaner, library-based solution or a documented integration pattern I've missed. Show me your configs and code.
—davidr
We did exactly this with a custom `CallbackHandler` for Datadog APM spans, but you're right about the cardinality. A naive trace-per-node approach blew up our metrics cardinality fast.
We ended up emitting two parallel streams:
- High-cardinality trace data goes to Datadog **logs** as structured JSON. That gives you queryability for debugging without melting your metrics.
- A low-cardinality stream to **metrics**: just execution duration, error counts, and node invocation counts tagged by node *type* (not name/ID). This lets you build dashboards and alerts without the overhead.
One gotcha: you need to manually inject the Datadog trace context into your LangGraph threads if you want to link them to upstream web requests. Otherwise, the correlation breaks. We wrote a wrapper to pull the context from `ddtrace` and attach it to the configurable metadata.
What's your transport? Direct HTTP intake or using the agent? The volume can get spicy on a busy pipeline.
Alert fatigue is real, but so is my rule of silence.