Skip to content
Notifications
Clear all

Walkthrough: adding custom metadata to all our Anthropic API calls

4 Posts
4 Users
0 Reactions
49 Views
(@fred99)
Estimable Member
Joined: 4 months ago
Posts: 95
Topic starter   [#29042]

I’ve been trying to better track our LLM costs and usage patterns by team. Since we use Anthropic’s API directly in several services, I wanted a way to attach project or team identifiers to every call without modifying each codebase.

I found Langfuse’s tracing can add custom metadata automatically. Here’s the basic integration I set up. You initialize the Langfuse SDK and wrap the Anthropic client. The key is using the trace params to inject a static metadata object.

This approach groups all spans under a shared trace name and attaches the custom data. Now our dashboards break down costs by team and project code. It was simpler than I expected.



   
Quote
(@coffeegoblin)
Reputable Member
Joined: 4 months ago
Posts: 352
 

That's neat until you need to switch off Langfuse or Anthropic changes their API. You've just traded one vendor lock-in for another, but now with a middleman.

What's your exit plan if Langfuse's pricing model shifts? I've seen these "simple" wrappers become mandatory infrastructure, and then you're stuck reverse-engineering all your metadata paths during a migration.

Also, static metadata? Hope you never need per-request or per-user tagging from that same service. That's when this tidy setup turns into a refactoring mess.


Buyer beware.


   
ReplyQuote
(@emilyk22)
Honorable Member
Joined: 4 months ago
Posts: 465
 

The static metadata approach you described is exactly what I've used for team-level cost allocation. It becomes problematic when you need to add dynamic tags, like individual user IDs for auditing, on top of the project codes.

Have you considered how you'll handle a scenario where a single service needs to make calls for two different internal teams? The wrapper will need logic to switch contexts, which starts to reintroduce the codebase modifications you wanted to avoid. I ended up using a thread-local storage pattern for this, which works but adds complexity.

What's your plan for capturing metadata from asynchronous or batch processing jobs, where the trace context might not propagate cleanly?


Support is a product, not a department.


   
ReplyQuote
(@helenr)
Honorable Member
Joined: 4 months ago
Posts: 534
 

You raise a fair point about vendor lock-in. A wrapper can become its own piece of critical infrastructure.

The exit plan for something like Langfuse, though, is often simpler than you'd think. Since it's just adding metadata and forwarding calls, you could replace it with a custom tracer that logs directly to your own storage. The real dependency is on the pattern itself, not necessarily the specific provider.

Static metadata is indeed a limitation, but it's a starting point for cost allocation, which seems to be OP's main goal right now. The refactoring you mention would be needed anyway if they later require per-request tags, regardless of whether they used a wrapper or modified each service directly.


—HR


   
ReplyQuote