We ran LangSmith for production LLM observability for about nine months. The data was useful, but the bill became a significant line item as our trace volume grew into the millions per month. The pricing model felt punitive for high-volume, low-margin use cases. We evaluated alternatives based on data ownership, cost structure, and core tracing/evals functionality.
We settled on Phoenix by Arize. The primary driver was the open-core model: we can run the OSS version on our own infrastructure. This eliminated the per-trace cost entirely. The managed offering is available if we don't want the operational overhead, but the pricing is more palatable.
**Key changes in our setup:**
* **Infrastructure:** Deployed Phoenix (OSS) on our existing EKS cluster. It uses OpenTelemetry under the hood.
* **Instrumentation:** Swapped LangSmith's SDK for Phoenix's. The change was minimal.
* **Data Pipeline:** Traces are now written directly to our own object storage (S3) and OLAP database (ClickHouse), not a vendor's system.
**Instrumentation diff (Python):**
```python
# Before: LangSmith
from langsmith import Client
client = Client()
# ... run tracing via decorators or context managers
# After: Phoenix
import phoenix as px
px.launch_app() # For local eval; in prod, traces export via OTLP
# Use OpenTelemetry instrumentation or Phoenix's trace decorators
```
**Results after 6 months:**
* **Cost:** LangSmith bill was averaging ~$1.2k/month. Current cost for Phoenix is the compute/storage for the EKS pods and ClickHouse, which is ~$180/month. This is a ~85% reduction.
* **Functionality:** We retained the essentials: trace visualization, latency/error tracking, and custom evaluation scoring. We lost some of LangSmith's integrated dataset management and playground features, which we didn't heavily use.
* **Operational Overhead:** Increased, but manageable. We now own the pipeline's reliability and scaling. This added perhaps 2-3 hours a week of DevOps attention.
**The trade-off is clear:** You exchange a turnkey SaaS for a self-managed OSS solution. If your team has the capacity to run it, the cost savings are substantial. If you're a small team that values pure SaaS, this migration is not for you.
Phoenix meets our needs for monitoring and evals. The UI is less polished, but it's functional. The ability to directly query the underlying trace data with SQL has been a significant advantage for custom reporting.
- cr
Your fancy demo doesn't scale.