I've been monitoring our production data ingestion pipelines that rely on Poe's API for the last 72 hours, and the latency profile has degraded significantly. This isn't a minor fluctuation; we're seeing sustained periods where P99 latency is 3-4x the baseline, with sporadic spikes that are an order of magnitude higher. Our alerting has been triggered multiple times, forcing us to implement circuit breakers and fallback logic.
Our setup is straightforward: stateless Python workers in Kubernetes making JSON POST requests to the chat completion endpoints. We've ruled out our own infrastructure. Network diagnostics show the increased latency is isolated to the time-to-first-byte from Poe's servers. The pattern doesn't correlate with our load.
Here's a simplified view of the latency histogram from our monitoring (Prometheus) for the `poe-api-request-duration-seconds` bucket over two days. The baseline mode was around 450ms. The problematic periods show a clear bimodal distribution.
```
# Metrics snippet showing the shift (values in milliseconds)
period : p50 | p90 | p99 | time_window
baseline (prior week): 450 | 720 | 1100 | stable
last_48_hours : 550 | 1200 | 3500 | volatile, with spikes >10000
```
The symptoms point to either internal service degradation or throttling that isn't being communicated clearly through HTTP status codes. We're receiving HTTP 200s, just taking much longer.
I need to know if this is a widespread issue or something isolated to our region/endpoint. Specifically:
* Are others observing similar latency spikes, particularly for the `api.poe.com` endpoints?
* Has anyone received any communication from Poe regarding performance issues or maintenance?
* What workarounds are you employing? We've had to batch requests and increase timeouts, which is suboptimal.
Without official status updates, we're left reverse-engineering their health from client-side metrics. For a paid API service, this level of opaque performance variance is problematic for any serious real-time application.
—davidr
—davidr