Just tried the new Bedrock integration. It's fine. About time, frankly.
The main gotcha is the pricing mapping. Langfuse's "unit" logic doesn't always line up with Bedrock's token count in a way that's clear. You'll think you're tracking one thing, but your cost reports might be off by a factor. Also, the anthropic models on Bedrock? The trace formatting gets weird. Spans within spans for no apparent reason.
Anyone else run into this, or just me?
CRM is a necessary evil
You're spot on about the pricing mapping. The unit logic is a leaky abstraction when you have a provider like Bedrock that bills per million tokens with different prices for input/output. Langfuse's attempt to normalize that into a single "unit" column for cross-provider comparison just doesn't hold water for accurate cost attribution.
I've seen the span nesting with Anthropic models too. It seems to happen when the model uses a tool calling flow. The parent span is the main invocation, but then you get an inner span for each tool call suggestion, even if the call isn't executed. It creates visual clutter in the trace viewer without adding much insight.
My workaround has been to add a post-processing step in the pipeline that flattens those traces before they hit our monitoring dashboards. Adds latency, but cleans up the view.
Extract, transform, trust
The unit mapping is a known issue if you're using tiered pricing models like Bedrock's. I ran a comparative test between the Langfuse-reported units and the actual token counts from Bedrock's CloudWatch logs for a batch of 10k invocations.
The variance was between 5-18% depending on the model family, because Langfuse's current multiplier doesn't account for the distinct input/output token pricing. For accurate cost tracking, you need to bypass the unit column entirely and calculate cost using the raw `input_tokens` and `output_tokens` fields from the trace, then apply the specific Bedrock price per million tokens for that region and model.
On the span nesting, can you confirm you're seeing this outside of tool use scenarios? The behavior user185 describes is expected for tool calls, but I've also observed it in standard completions when the trace includes streaming.
Show me the numbers, not the roadmap.
That unit mismatch is worse than just a mapping issue. The Langfuse SDK's default cost calculation for Bedrock uses a generic tokens-per-unit constant, which is wrong for any model with different input/output pricing.
If you're on the Claude 3 family, you need to override the cost calculation in your instrumentation. I dug into the Python SDK and you can pass a custom `cost_calculator` function that uses the specific `InputTokenPrice` and `OutputTokenPrice` from the Bedrock model card. Without that, your unit column is useless for actual spend.
The span nesting, yes. It's not just Anthropic. The integration seems to create a wrapper span for the entire Bedrock invocation, then child spans for each individual message in the request array. If you're batching or have a system prompt separate from the user message, it explodes the trace depth. Makes Jaeger unreadable.
shift left or go home
Yeah, I saw that about the pricing mapping too. Makes the unit column kind of misleading for actual cost tracking.
A follow-up question, since you mentioned it: for the weird span nesting, are you seeing that even when you're not using tool calling? I'm about to test the integration with a simple prompt, and I'm wondering if I'll need to adjust my dashboards right away.
The unit mismatch is the whole point, isn't it? They built a metric called "unit" that can't be used for its intended purpose. Calling it a "gotcha" is generous; it's a fundamental error in abstraction for any provider with tiered input/output pricing.
The span nesting isn't just visual clutter. It breaks duration calculations for the actual model call if you're relying on the parent span timing. You're measuring the wrapper overhead, not the inference.
Data skeptic, not a data cynic.
Yeah, the pricing thing is a real issue. I'm just getting started with Langfuse and I was actually counting on that unit column to estimate my AWS bill for Bedrock usage.
> weird span nesting
I'm curious if you saw this on a single, simple prompt? Or only when you had a more complex message array? Trying to figure out if I need to budget time to fix my trace views right away.
If you were counting on the unit column for your AWS bill, stop now. It's wrong for Bedrock.
For span nesting with a single prompt: yes. I saw it with a basic `claude-3-sonnet` call. The integration creates a parent "bedrock invocation" span and a child span for the actual request. That child span's duration is what you want, not the parent's. You'll need to adjust your views.
—cp