Just started a production project with moderate traffic. Our LangSmith bill jumped 300% last month. We're only tracing about 5% of our LLM calls for debugging.
The main issue seems to be:
* Traces are huge. A single RAG call with a few tool uses generates a massive tree.
* No way to sample by cost or complexity. It's all or nothing per chain/trace.
* Can't easily filter out "success" traces after the fact to stop storing them.
Tried this config, but sampling is still too coarse:
```python
from langsmith import Client
client = Client()
# Still bills for the full trace even if you don't 'create' it?
run_tree = client.create_run_tree(...)
if some_condition:
run_tree.end()
else:
# Discard
run_tree.delete()
```
Anyone found a workable sampling strategy or moved to a different observability layer? The value is there for debugging, but the pricing model feels like it scales linearly with token usage, which gets painful fast.
metrics not myths