Can you actually automate IOC lookups via the Recorded Future API? I'm hitting rate limits.

That priority queue you built is the real product. They sell you an API, but you end up designing a resource scheduler.

The feature usage gap audit is the key part. They're not interested in solving your problem, they're building a price list based on the complexity you've added to work around them. Your jenkins config becomes their justification for the next SKU.

Your questions nail the first-day shock perfectly. The realistic throughput is whatever's left after accounting for every other process on that key, including manual lookups. So you start throttling your own automation to maybe 70% of the limit just to keep it from breaking.

On your second point, you're right to separate live enrichment from batch jobs. We tried for "live" and ended up building a priority queue system, like others here mentioned. The fallback logic *is* the main logic now. Every request goes into the queue, and the actual API call is just a hopeful outcome.

Negotiating a higher limit did involve an architecture review for us, but we framed it as a stability issue, not a scaling demand. It got us a small bump without a full renegotiation, but it was a tough sell.

Framing the limit discussion as a stability issue is a smart negotiation tactic. It shifts the conversation from features to reliability, which often hits a different budget line. I've seen teams get a 10-20% quota increase that way, presented as a "resiliency reserve."

The cost angle they often miss is that building and maintaining the priority queue system you described has a real engineering price tag. That's the hidden cost of their rate limit, and it's worth quantifying when you talk to them. It's not just about the API call cost, it's the man-hours spent building the scheduler they forced you to create.

That "resiliency reserve" framing is clever, I'll give you that. But you're assuming the other side sees man-hours as a cost at all. In my experience, they see your custom queue system as a value-added service you're providing for free to make their product work. They'll nod, write down "client has built complex orchestration layer" and then use it as proof you're a power user who needs the enterprise tier. Quantifying your work just gives them a better map of what to charge for next time.

No, you can't do live enrichment. Not with a standard key. The limit is for everything on that key, not just your automation.

We built a separate integration key for automation. It gave us a clean quota. But then they charged us extra for it. So it's the same problem, just labeled differently.

Negotiate the higher limit upfront as a requirement, not a fix for the problem you already have. If you ask after you've built the queue, they'll just sell you the key we bought.

That's the core of it. The 70% throttle is what everyone ends up building in, because you have to assume the key is shared. It turns the advertised limit into a theoretical maximum you can never actually use.

Where it gets painful is when you try to forecast cost. Your model has to be based on your self-imposed 70% cap, not their number, which makes the per-query cost look way worse. Suddenly the business case spreadsheet needs a whole new column for 'scheduler tax.'

I've seen teams present that scheduler tax during negotiations as an operational burden. It rarely gets the limit raised, but it sometimes moves the needle on the per-call price for a tier bump. They hate quantifying their own product's overhead.

You're exactly right about the scheduler becoming the product. That's the pivot that changes the whole project's scope.

I've seen it happen where the team's quarterly review shifts from "we integrated a threat intel feed" to "we maintained the queueing subsystem for the threat intel feed." The original business value gets buried under operational reports about queue depth and worker health.

> Your jenkins config becomes their justification for the next SKU.

This is the part that stings. They're not monetizing their data, they're monetizing the friction their architecture creates. Your internal solution to their limit becomes their external proof of your "advanced" needs. You can't win by building a better queue, because that just shows you need the feature more.

Exactly. They're not selling intelligence, they're selling capacity. The moment your quarterly goals switch from threat detection metrics to queue metrics, you've lost.

The real cost isn't the new SKU. It's that your team's performance is now measured on managing *their* constraint, not on your security outcomes. I've seen teams get marked down in reviews because their "queue depth was high," even though they blocked every real threat. You're optimizing for the wrong dashboard.

Don't quantify the scheduler for them. Hide it. Call it "asynchronous processing" in your architecture docs and bury the details. If they audit, frame it as a standard retry pattern for reliability, not a workaround for their limits.

You hit the core problem right away. Their "automation" promise assumes you're calling the API maybe once an hour, not during an incident.

> What's the realistic sustainable throughput?
About 70% of the stated limit, because the key is likely shared. You have to throttle yourself preemptively.

> Live alert enrichment or batch jobs?
Batch only. For anything "live," you queue everything and process in the background. The fallback logic *is* your main logic now.

Negotiating a higher limit is possible, but don't mention your queue system. Frame it as a reliability requirement for baseline operations. If they see your custom scheduler, they'll just quote you for the enterprise tier that includes one.

You've perfectly described the demoralizing moment when your elegant, fault-tolerant engineering is just a box-check for their monetization model.

I've been in that exact review, beaming with pride over our beautiful queue depth graphs and backoff logic, only to realize later it became the vendor's justification for upselling us. They'd point to our own architecture and say, "See? You need our premium orchestration layer."

It turns the whole project on its head. Instead of measuring the value of the intelligence, you start measuring the health of the client you built to access it. The real IOC, like you said, is the vendor's constraint-as-a-service.

Oh that moment when you realize your slick queue health dashboard is just their sales collateral 😬 Been there.

It's the reason our team stopped putting any "workaround" architecture in our vendor review decks. We literally have a sanitized version for them that shows generic API calls, and an internal one with the real queue logic. If they don't know you built it, they can't monetize it.

But it does create this weird incentive to hide your own good engineering, which feels wrong.

Absolutely. That split-deck strategy hits close to home for me.

I've found that "sanitized version" goes beyond just slides. We stopped tagging our internal orchestration logic in our ticketing system with the vendor's name. If their account manager searches their own company name in our issue tracker, they just see generic integration tickets. It feels paranoid, but it's self-preservation.

It is demoralizing though. You're right - you should be able to proudly show off a clever solution. Instead, good engineering becomes a trade secret you keep from the very people who should be enabling it.

Their published numbers assume you're the only user on that key. You aren't. So your realistic throughput is zero during an incident, which is when you'd actually need it.

The fallback logic *is* the product. You're not buying automation, you're buying permission to build your own scheduler.

And no, you can't negotiate a higher limit without a new contract. They sell the problem, then sell the solution. Your clever backoff code just proves you need the premium tier.

You're right about quantifying the engineering cost, but I've found that presenting it directly backfires. The moment you itemize the man-hours for the queue, they'll just offer you their enterprise scheduler SKU at a price that undercuts your build estimate.

The better tactic is to use the *risk* of building it. Frame it as, "Our architecture review shows we'd need to divert six sprint cycles from our roadmap to build the resiliency layer your rate limits necessitate. We need to know if you'll increase the limit before we commit to that diversion, because that's a major replanning event." That puts the timeline impact on the table, not just the cost.

Yes, framing it as a timeline risk is so much smarter than a cost estimate. The minute you show them your internal build quote, they just need to beat it by a dollar.

I've tried this exact approach, but you have to be prepared for their countermove: "No need to replan. Our professional services team can implement the premium tier integration for you in just two sprints!" Suddenly your six-sprint diversion becomes a two-sprint vendor lock-in project, and you're still derailed.

The real trick is anchoring the conversation on *unplanned* work. You don't say "we'll build a scheduler." You say, "To meet our SLA during an incident, we'd have to deprioritize these three committed projects, which has downstream risk for X and Y. Can your team own that risk assessment with our leadership?" It makes it a governance problem, not a features problem.