Skip to content
Notifications
Clear all

Help: SDK tracing setup for Node.js is causing huge latency spikes

2 Posts
2 Users
0 Reactions
5 Views
(@jenniferg)
Estimable Member
Joined: 2 weeks ago
Posts: 76
Topic starter   [#12175]

Hi everyone. I've been working with Freeplay's Node.js SDK for a few months now to trace LLM calls from our backend services. Overall, the insights have been valuable, but I've hit a major blocker.

We're seeing intermittent but severe latency spikes—sometimes adding 2-3 seconds to a request—that we've traced directly to the Freeplay tracing calls. Our setup is fairly standard: we're instrumenting a multi-step RAG workflow using the `wrapOpenAI` client and sending custom traces for our own functions. The spikes don't happen on every request, which makes it particularly frustrating to debug.

Here's a simplified version of our initialization:

```javascript
const freeplay = require('@freeplayai/freeplay');
const openai = new OpenAI();

const freeplayClient = freeplay.init(apiKey);
const wrappedClient = freeplayClient.wrapOpenAI(openai);
// ... then using wrappedClient for chat.completions
```

We've followed the async tracing patterns in the docs, but the latency seems to occur somewhere in the batched event submission. Has anyone else encountered this? I'm trying to determine if this is a configuration issue on our end—perhaps with batch sizes or timeouts—or a known issue with the SDK under certain loads.

We're running Node 18 on a fairly robust Kubernetes setup, so resource constraints aren't likely the root cause. Any insights into optimal config for high-throughput B2B applications, or even workarounds like disabling automatic tracing for certain endpoints, would be hugely appreciated. I'm hoping to balance the need for observability with our strict latency SLAs.

— jg


Let's keep it real.


   
Quote
(@crm_hopper_2028)
Reputable Member
Joined: 3 months ago
Posts: 135
 

Yeah, those batched submissions can be a real pain when they decide to hang. I've seen similar things with other APM SDKs.

A couple things to try that helped for me:
- Lower the batch size dramatically, like to 5 or 10, and see if the spikes become more frequent but smaller. That can point to a network or processing bottleneck on their end.
- Check your `timeout` config for the Freeplay client. If it's too high, it might be waiting a long time for a batch to fill up before giving up and moving on. A shorter timeout can force failsafes to kick in.

Have you looked at the actual network traces for when the spike happens? It's probably sitting on a `POST` to their ingest endpoint.


Still looking for the perfect one


   
ReplyQuote