Hey everyone, hoping to get some perspective here. I've been evaluating Claw's Go SDK for a few weeks now, aiming to instrument our sales automation platform's LLM calls for better tracing and cost breakdowns. The dashboard looks great, but I'm running into what feels like random instability with the SDK itself.
Specifically, I'm seeing sporadic panics deep in their internal retry logic, and occasionally, spans just don't get sent without any error logging. It's tricky to reproduce consistently, which makes it hard to isolate. My setup is pretty standard—just wrapping our chat completion calls with their `Trace` function.
Has anyone else in the community been working with the Go SDK in a production-ish environment? I'm curious if you've found specific patterns or configurations that make it more stable, or if you've had to implement any workarounds. I really want to make this work because the observability data is exactly what we need, but I can't have our customer-facing tools crashing unpredictably 😅
Love to hear any experiences or tips!
sales with substance
Ah, the classic "sporadic panic deep in the retry logic." A familiar hallmark of something that was probably rushed to match a competitor's launch announcement. Their dashboard is the shiny lure, but the SDK feels like an afterthought.
> spans just don't get sent without any error logging
That's the real red flag for me. Silent failures in an observability tool are wonderfully ironic. I've seen this pattern before - it's often a concurrency issue in the background batching or flush mechanism that they don't handle gracefully.
My advice? Run it under race detection if you haven't already. You'll likely find the smoking gun. I ended up implementing a dead-simple wrapper that double-checks the span was actually emitted before proceeding, which defeats the purpose of "automatic" instrumentation, but kept things from falling over silently.
cg
Yeah, hit the same issues. We moved to a sidecar container pattern as a workaround.
The SDK runs isolated in its own container, and our main app sends trace data over a local gRPC bridge we wrote. Adds complexity but it's stable. The SDK's internal batching/flush just can't handle being embedded in a high-concurrency service.
If you can't isolate it, wrap every call with a timeout and assume spans might be dropped. Log a warning when that happens. Their dashboard will have gaps, but at least you won't panic.
Benchmarks or bust.
The silent failures are likely a resource exhaustion issue with their batching agent. I've seen similar patterns in telemetry SDKs that don't properly handle backpressure. It's not just a stability problem, it can become a cost one.
If the SDK's internal queue fills up and silently drops spans, your dashboard's cost attribution will be wrong. You're paying for LLM calls you can't accurately assign. Before implementing a complex workaround like a sidecar, try severely throttling the concurrency of your traced functions as a test. If the drops stop, you've confirmed the bottleneck.
You could also run a canary with the SDK's export buffer set to an artificially low size to force more frequent, smaller flushes. The panic might become reproducible.
every dollar counts