Hitting a wall with Mandiant's API rate limits while trying to enrich security events. My script batches IOCs from our SIEM, but the throttling is causing major delays and timeouts in our alert triage workflow.
Has anyone built a reliable pattern for this? I'm looking at:
* Implementing a queuing system with exponential backoff
* Caching results locally to avoid repeat lookups
* Maybe distributing calls across multiple API keys (if allowed)
What's been your most effective fix? The goal is keeping the automation fast without getting blocked. Thanks for any real-world tips.
--ash
data over opinions
Caching is your highest-impact move here, especially if your IOC data has reasonable TTL. I've seen scripts cut external API calls by 60% with a simple Redis cache using the IOC as key. Make sure to respect Mandiant's data freshness requirements though.
Queuing with exponential backoff is the standard pattern for handling 429s, but you need to distinguish between hard rate limits and temporary throttling. Your backoff logic should be adaptive.
Using multiple API keys is risky if it violates their ToS - that's a quick way to get all your keys revoked. I'd focus on optimizing the single-client path first. Have you checked if their API supports batch endpoints? Sending 10 IOCs in one call is far better than 10 calls.
sub-100ms or bust
The batch endpoint suggestion is spot on - most security vendors have them buried in their docs. Mandiant's v4 API actually has a `/indicators/attributes` batch endpoint that'll take up to 1000 IOCs per request. Found that after digging through their GitHub issues.
But your caching TTL warning is crucial. We got burned once using a 24-hour cache for malware hashes, only to find out a known-bad file got reclassified as clean after 8 hours. Now we tie our Redis expiry to the `last_updated` field in the API response, if present. Otherwise it's a risky guess.
Distributing across API keys is a total minefield. Better to negotiate a rate limit increase with their sales team if you're hitting walls - they're often flexible if you have a legit use case.