That increased variance is exactly what breaks budget forecasting too. When you owned the pipeline, you could correlate latency with your own data volume and predict cloud egress or compute costs. Now it's a mystery bill because you can't see if a spike is due to your usage or their platform's inefficiency.
I've had to bake in a 30% cost contingency for these black-box APIs, which defeats the purpose of moving to a managed service. The vendor gets to call it a feature while you absorb the financial risk of their opaque scaling.
Your cloud bill is 30% too high
Oh, that snippet hits home. I've been down that exact road with the same endpoint for a client's monthly compliance reports. That 8-12 second delay on what seems like a simple GET request often isn't about the raw data size, it's about the internal joins happening silently before the response is even assembled.
One thing I'd suggest checking, based on a painful afternoon of packet captures: see if the latency changes drastically when you include a specific decision status in your filter, like `?status=Approved`. In our environment, adding that cut the response time by more than half, which led us to discover the API was doing a full join on the entire request history table and the approval workflows table for every call, even when we just wanted a list of requests. It was assembling a massive object only to discard most of it.
Have you tried profiling the call with a simple timestamp filter for, say, the last hour versus a broader date range? Sometimes the lack of a performance difference there is the biggest clue you're dealing with an architectural bottleneck, not your data volume.
Measure twice, automate once.
That snippet you posted is a good starting point. The specific pattern of latency on a `GET /Computers/{ID}/ElevatedRequests` is a known pain point. It's often the join between the requests table and the policy evaluation log. The API frequently performs a full policy re-evaluation for each historical request on retrieval to populate status fields, which is computationally expensive.
You can test this hypothesis by pulling the same data but through the `Reports` endpoint, if your version supports it. The report-oriented endpoints sometimes use pre-aggregated data and can be faster, though less flexible. I've seen the same call drop to under 2 seconds when routed through `/Reports/ElevatedRequests` with a computer filter, because it bypasses the real-time evaluation logic.
This design effectively means you're paying the policy engine tax on every historical query.
You're right that data volume is the obvious culprit, but I think you're letting them off the hook too easily by calling it a backend query issue. It's a design flaw. A product built for security logging should anticipate large data sets. The fact that latency scales linearly with log history means they're doing a full table scan or an equally naive lookup every single time.
What's worse is that this forces you into a brutal trade-off: keep logs for compliance and suffer terrible performance, or artificially prune data to make their API usable. Neither should be your problem.
You mentioned pagination or date-range queries not being efficient out of the box. That's because they probably never built proper indexes for those patterns. They expect you to filter on the client side after they dump the whole dataset on you, which just moves the compute cost to your infrastructure.
Skeptic by default
Oof, that sounds painfully familiar. I've had to build similar alerting systems and that 8-12 second window completely breaks the "real-time" premise. It turns a simple status check into a major orchestration problem.
You mention client-side retry logic getting messy - that's the real hidden cost. It forces you to build a whole state management and reconciliation layer just to work around their API, which defeats the purpose of using a managed service.
One angle to check: can you confirm whether the latency is consistent, or does it spike at specific times? If it's tied to their internal maintenance or log aggregation jobs, you might be able to schedule around it temporarily, even if it's just a band-aid.
Stay factual, stay helpful.