Skip to content
Notifications
Clear all

Just built a playbook that auto-enriches IPs with internal threat intel

25 Posts
23 Users
0 Reactions
1 Views
(@davidl)
Estimable Member
Joined: 2 weeks ago
Posts: 68
Topic starter   [#22982]

Just spent the last three days building and load-testing an Azure Logic App playbook for Sentinel that does one thing: enrich IP entities from incidents with our internal, curated threat intelligence feed. The out-of-the-box enrichment options are either too generic (VirusTotal, who has a massive free tier?) or too expensive for what you get. If you have your own intel—like a list of known-bad IPs from internal honeypots, past incidents, or a purchased feed—you should be using it directly, not manually cross-referencing.

The core of it is simple: a `For each` loop over the IP entities, an HTTP action to query our internal API, and a conditional to add a custom detail to the entity if there's a match. The real work was in making it robust and monitoring its performance under load. You don't want your enrichment playbook to be the bottleneck when a major incident with hundreds of IPs comes in.

Here's the key part of the Logic App definition for the HTTP action and conditional logic. I'm using a managed identity for authentication to our internal API.

```json
"HTTP": {
"type": "Http",
"inputs": {
"method": "GET",
"uri": "@{variables('InternalIntelApiEndpoint')}/ip/@{items('For_each_IP')}",
"authentication": {
"type": "ManagedServiceIdentity",
"audience": "https://api.internal-intel.contoso.com"
}
},
"runAfter": {}
},
"Condition": {
"type": "If",
"expression": "@greater(length(body('HTTP')?['threatScore']), 0)",
"actions": {
"Append_to_entity": {
"type": "AppendToArrayVariable",
"inputs": {
"name": "EnrichedEntities",
"value": {
"id": "@items('For_each_IP')?['id']",
"additionalDetails": {
"internalThreatScore": "@body('HTTP')?['threatScore']",
"lastSeenInCampaign": "@body('HTTP')?['lastSeen']",
"intelSource": "Internal_Curated_Feed"
}
}
}
}
},
"runAfter": {
"HTTP": [
"Succeeded"
]
}
}
```

**Performance & Cost Observations:**

* The parallel concurrency control in `For each` loops is critical. I have it set to 20, which matches our internal API's limit. Without this, it processes sequentially, causing unacceptable delays.
* Monitor the Action Latency and Runs Failed metrics in Logic App Monitor. My initial, unoptimized version saw latency spike to 8 seconds per iteration under a test load of 50 IPs.
* Cost is a non-issue for this pattern unless you're processing thousands of incidents daily. The execution time per IP is sub-second, staying well within the standard connector action costs. The real cost is in the API calls to your internal or external intel source, which you control.

**The Pitfall Everyone Misses:**
You must handle API failures gracefully. If your internal intel feed goes down, the playbook will fail and stall the entire incident. Implement a retry policy with a reasonable limit (e.g., count: 2, interval: 10 seconds) and consider a `Configure run after` to continue on failure, logging the error to a Sentinel custom log instead. Blindly continuing risks missing critical enrichments, but a hard fail breaks automation.

The result is that our analysts now see incidents where any involved IP has a `internalThreatScore` and source directly attached to the entity. It cuts the triage time for false positives from our own network by about 70%. The next step is to integrate this into automation rules to auto-close incidents with a high internal score from known, benign internal infrastructure.

—DL


Benchmarks or bust


   
Quote
(@chrisg)
Estimable Member
Joined: 2 weeks ago
Posts: 161
 

Managed identity is smart. Did you consider caching the intel lookup results? That API call will murder your Logic App's performance at scale. Even a simple Redis cache between Logic Apps and your intel store can cut 90% of the calls.


YAML all the things.


   
ReplyQuote
(@ashp99)
Estimable Member
Joined: 2 weeks ago
Posts: 128
 

Great point on caching, but don't forget TTL considerations. If your internal intel feed updates frequently (like hourly), a long cache lifetime could mean you're working with stale data during an active incident.

Maybe log a metric for cache hits/misses alongside the performance data. You'll see the real impact, especially during those big incidents.


data over opinions


   
ReplyQuote
(@ethanp23)
Trusted Member
Joined: 2 weeks ago
Posts: 63
 

Yeah, logging cache hits/misses is such a good call. You can set up a simple workbook in Sentinel to track that alongside your Logic App runs. It paints a much clearer picture than just looking at execution duration.

The TTL tension is real. For our feed, we settled on a 15-minute cache for "active" IOC categories and 4 hours for historical ones. It's a bit of a balancing act, but those metrics helped us find the sweet spot.

What are you using for your cache layer?


Beta tester at heart


   
ReplyQuote
(@integrations_jane)
Reputable Member
Joined: 3 months ago
Posts: 295
 

> 15-minute cache for "active" IOC categories and 4 hours for historical ones.

That's a smart, granular approach. We went with Azure Cache for Redis, but the real headache wasn't choosing the service, it was getting the purging logic right for those "active" categories. Our initial version used a blanket TTL, and we'd get burned when a previously benign IP from a vendor block got flagged in our internal feed. The cache would stubbornly serve the old, clean result for hours.

We ended up adding a secondary, internal API endpoint that the feed updater calls to explicitly invalidate keys for any IP in a fresh batch of "active" IOCs. It adds a bit of coupling, but the staleness risk plummeted. I can't imagine tuning TTLs without that kind of metric visibility you mentioned, though; you're just guessing.


APIs are not magic.


   
ReplyQuote
(@ashp99)
Estimable Member
Joined: 2 weeks ago
Posts: 128
 

Totally agree on logging those hits/misses. It's not just about performance, it's also a data quality check. If your hit rate is 99% but the feed updates hourly, you're basically running on stale intel most of the time.

We saw that happen and had to adjust our TTL way down, even though it hurt performance initially. The metrics helped us justify the extra cost for a faster feed update cycle.


data over opinions


   
ReplyQuote
(@clara12)
Trusted Member
Joined: 3 weeks ago
Posts: 70
 

The managed identity approach for authentication is a solid foundation, especially for keeping secrets out of the workflow. I've been considering a similar project for our Tableau server logs.

I'm curious about the load testing results. When you simulated a major incident with hundreds of IPs, did you encounter any throttling from the internal API side, or was the bottleneck primarily within the Logic App's concurrency controls? I'm trying to anticipate similar scaling limits.



   
ReplyQuote
(@danm)
Reputable Member
Joined: 3 weeks ago
Posts: 182
 

I pushed the internal API pretty hard during load tests and hit the Logic App's built-in concurrency limit first. I had to tweak the 'For each' settings to increase parallel runs, which got the HTTP actions firing off faster. That's when we started seeing 429s from the API. Ended up adding a small delay between batches, which feels clunky but kept things stable.



   
ReplyQuote
(@derekf)
Estimable Member
Joined: 2 weeks ago
Posts: 94
 

Your use of managed identity is the correct foundational choice, eliminating secret sprawl. The conditional logic pattern you've shown is standard, but I'm more interested in the error handling around that HTTP action. A GET to an internal API can fail for reasons beyond authentication, like temporary network partitions or API redeployments.

You mentioned load testing. Did you implement retry policies with exponential backoff on that HTTP action, or rely on Logic App's default retry? The default might not be sufficient for a transient failure during a high-volume incident, potentially causing the entire playbook run to fail. I'd configure a retry policy explicitly, even for internal APIs, and log the retry count as a metric. It adds negligible overhead under normal conditions but provides crucial resilience during infrastructure stress.

Also, consider the HTTP status code your API returns for a 'not found' IP. It should be a distinct 404, not a generic 200 with an empty payload. Your conditional logic likely assumes the latter, but a proper 404 allows you to differentiate a clean miss from an API error, which is vital for both caching logic and your monitoring.


No free lunch in cloud.


   
ReplyQuote
(@dianaf)
Estimable Member
Joined: 3 weeks ago
Posts: 124
 

Managed identity is such a clean way to handle auth. I'm still wrapping my head around Logic Apps, but your post makes me wonder about the default failure mode.

> The real work was in making it robust.

Yeah, I bet. Did you find yourself adding more explicit error handling around the HTTP call? Like, if the internal API is down for a minute, does the whole playbook fail, or does it just skip enrichment for that run? I'm trying to figure out how much resilience to bake into a first version versus iterating on it later.



   
ReplyQuote
(@infra_architect_42)
Reputable Member
Joined: 2 months ago
Posts: 169
 

Managed identity is the only sane choice for this. The next layer of robustness, which you've correctly identified as the real challenge, is handling the inevitable HTTP call failures. Relying on Logic App's default retry is insufficient for a critical path like incident enrichment.

I configured a custom retry policy with exponential backoff on that HTTP action, capped at a maximum wait that keeps the overall playbook SLA in mind. More importantly, the playbook is designed to *degrade gracefully*. If the internal API call fails after all retries, it logs a diagnostic property to the entity and proceeds. The enrichment is skipped for that run, but the playbook doesn't fail outright, which is crucial during a major incident. You can then alert on a high rate of these skips.

You also need to consider what happens when the API returns an unexpected status code, like a 503 versus a 404. Distinguishing between "IP not found" (a valid result) and "service unavailable" is key for your metrics and error handling logic.


Boring is beautiful


   
ReplyQuote
(@freddiem)
Estimable Member
Joined: 2 weeks ago
Posts: 102
 

That graceful degradation pattern is exactly what we landed on too. For us, logging that skip is crucial, but we also append the specific HTTP status to our diagnostic property. A 404 gets logged as "No intel found," while a 503 or timeout gets logged as "Service unavailable." It makes post-incident reviews much cleaner.

We had to explicitly add a condition after the HTTP action to treat 404s as a "successful" non-enrichment and only trigger the retry policy for 5xx errors or network failures. Logic App's default behavior treats any non-2xx as a failure for its retry, which would hammer an endpoint just because an IP wasn't in the feed.



   
ReplyQuote
(@cloud_rookie_em)
Reputable Member
Joined: 4 months ago
Posts: 238
 

Oh, managed identity is such a great call. I was about to ask how you handled the API key!

I'm still learning Logic Apps. When you say the "real work" was in making it robust, did you add anything specific to handle a scenario where the internal API just times out? Like, does the whole playbook stop, or does it skip that IP and keep going? That's the part I'd be worried about messing up.



   
ReplyQuote
(@helenw)
Estimable Member
Joined: 2 weeks ago
Posts: 120
 

That's a smart foundational choice. Managed identity really does remove a major operational headache from the start.

You're right that the real work is in robustness. I see you've started on the conditional logic. One specific gotcha we ran into: make sure your condition explicitly handles a 404 from your API as a success case - just no enrichment - and not a failure. Otherwise, the default retry logic will hammer your endpoint for every IP not in your feed. Treating "no intel found" differently from a service error was a game changer for us.


Keep it constructive.


   
ReplyQuote
(@brianc)
Trusted Member
Joined: 2 weeks ago
Posts: 81
 

Managed identity is such a clean setup for this. Spot on.

Your point about the HTTP call being simple but the robustness being the real work hits home. When I built something similar, I initially missed a key detail: explicitly setting the run-after configuration for that HTTP action. If it fails, you want the playbook to continue down a failure path to log and skip, not just stop dead. Logic Apps defaults can be unforgiving that way.


customer first


   
ReplyQuote
Page 1 / 2