After six months of using LangSmith as the central observability layer for our LLM pipelines, my team has completed a migration to OpenTelemetry for tracing, with DVC (Data Version Control) for prompt/response versioning. The primary driver was vendor lock-in and the escalating cost of storing traces at LangSmith's granularity. Our data is now portable, stored in our own cloud object storage, and our monthly observability costs have dropped by approximately 65%.
The architecture hinges on two key components:
1. **OpenTelemetry Tracing:** We instrumented our LangChain and custom Python services using the `opentelemetry-sdk` and `opentelemetry-exporter-otlp`. Traces are exported to a collector, which then batches and forwards them to a dedicated tracing backend (we chose Grafana Tempo, but Jaeger is equally valid).
2. **DVC for Artifact Versioning:** Instead of logging every individual prompt and response as a trace event detail, we now hash and version them as data artifacts. DVC manages the pointer files in our Git repo, while the actual JSONL files reside in S3.
A simplified version of our instrumentation looks like this:
```python
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
tracer_provider = TracerProvider()
trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)
# Export to OTLP collector
otlp_exporter = OTLPSpanExporter(endpoint="http://otel-collector:4318/v1/traces")
span_processor = BatchSpanProcessor(otlp_exporter)
tracer_provider.add_span_processor(span_processor)
with tracer.start_as_current_span("llm_invoke") as span:
span.set_attribute("llm.provider", "openai")
span.set_attribute("llm.model", "gpt-4")
# Log the input/output via DVC artifact ID, not as span attributes
span.set_attribute("artifact.input", "dvc://prompts/v1/abc123.json")
span.set_attribute("artifact.output", "dvc://responses/v1/def456.json")
```
**Quantifiable outcomes vs. LangSmith:**
* **Cost:** Reduced from ~$450/month (LangSmith project tier + overage) to ~$160/month (S3 storage, Grafana Cloud Tempo/Logs bundle).
* **Query Performance:** Trace aggregation queries (e.g., average latency by model) are now SQL queries against trace data materialized in our warehouse (using the Tempo -> Loki -> ClickHouse flow). Complex filtering is slower than LangSmith's optimized UI, but acceptable for our batch analysis needs.
* **Portability:** All trace data and versioned artifacts are owned by us, in our own infrastructure. We can switch visualization or tracing backends without a data migration project.
The trade-off is immediate, interactive debugging. LangSmith's UI for jumping between traces and inspecting detailed inputs/outputs is superior. We've partially mitigated this by building a simple internal tool that fetches artifacts via DVC based on trace IDs. This setup isn't for everyone, but if your primary needs are cost control, data ownership, and integrating LLM observability into your existing data stack (dbt, BI tools), the OTel+DVC path is worth serious consideration.