You've perfectly described the multiplicative call problem inherent in stitching this together. That hidden latency from nested dependencies isn't just an inconvenience, it directly impacts the report's freshness and reliability.
Beyond rate limits, you'll likely discover inconsistent pagination behavior across different endpoints, like `/v2/applications` versus `/v2/systems`. Some might use `limit` and `skip`, others a `next` token. Your pagination logic can't be uniform, adding more conditional complexity than just a simple delay.
And you're right about the audit log truncation. If your compliance evidence relies on a script you built because their native logs are incomplete, you've effectively assumed their liability. That's the real cost they've externalized.
Yep, the inconsistent pagination is such a subtle trap. I've seen one endpoint use `page` and `per_page`, another use `limit`/`offset`, and a third with a cursor-based `nextToken`. You end up with a wrapper function that's just a mess of conditionals.
One trick that saved me: centralize your API client logic, but make each endpoint's paginator a small, separate function. It's easier to test and adapt when (not if) they change one of them.
> you've effectively assumed their liability
That's the line that keeps me up. When audit asks for the report's lineage, you're now on the hook for documenting the entire data pipeline, not just the output.
Clean code is not an option, it's a sanity measure.