We're running a multi-service RAG app. Each LLM call triggers a chain across:
* embedding service
* vector DB
* orchestration layer (LangChain)
* the LLM provider itself (Azure OpenAI)
Our billing is a single, opaque monthly Azure OpenAI bill. We can't tell which downstream service or tenant is driving the cost.
Has anyone solved this? We need to attribute costs per call, per service, and ideally per internal customer.
What we've tried:
* Added custom metadata via Azure's `extra_body`. It's spotty.
* Built a proxy to inject headers. Adds latency.
Considering:
* OpenTelemetry tracing with cost dimensions in spans.
* A sidecar that logs token usage per request.
Need something that works with:
```yaml
# Our current GitHub Actions deploy pattern
- uses: azure/aks-set-context@v3
- run: kubectl apply -f tracing.yaml
```
What's actually working in production for you?
Ship it, but test it first