Hey everyone! 👋 I've been lurking here for a while, learning a ton from you all. I'm a project manager who's always trying to find better ways to keep our remote team's workflows healthy, so I'm super interested in tools that help with observability and catching weird stuff before it becomes a problem.
I was playing around with Traceloop's streaming API over the weekend, and I managed to build a little anomaly detector for our LLM calls. It's basically a small service that listens for spikes in latency or errors in specific workflows we run through our tools. I'm still new to this whole observability space, so it's probably super basic, but I was really excited that I got it working!
Here's a simplified snippet of how I set up the listener (I hope this is okay to share):
```python
# This is the core part where I listen for events
from traceloop.sdk import Traceloop
def handle_stream_event(event):
if event.latency > 2.0: # threshold for our use case
send_alert_to_slack(f"High latency detected: {event.name}")
Traceloop.streaming_client.on("span", handle_stream_event)
```
It's been running for a couple of days and already flagged a recurring slow call we didn't know about! I'm wondering if anyone else has tried building custom monitors with their streaming features? I'd love to hear how you're using it or if there are any pitfalls I should watch out for as I expand this. Maybe there's a better way to structure this?
Thx!