I've been working on integrating Mandiant's Threat Intelligence feed into our SOAR platform, and I ran into a common issue: the JSON structure from their API can be a bit... inconsistent for automation. Specifically, the way some fields are nested or formatted made it tricky to map directly to our playbook triggers.
After spending more time than I'd like to admit on it, I wrote a small Python normalizer. It flattens the key indicators and metadata we care about (like malware families, associated actors, and CVEs) into a more predictable schema. It's been working well for our last few ingestion runs.
I thought others here might be dealing with the same thing. The script is pretty basic, but it handles the main object types we pull—malware, actors, and vulnerabilities. It uses a few simple mappings and cleans up some of the list-of-dicts formats into plain strings or arrays.
You can find the gist here: [link redacted]. It's set up to work with the `v4` feed. The main function is `normalize_mandiant_item`. If you give it a raw JSON object from the API, it returns a flattened dictionary with fields like `name`, `type`, `industries_targeted` (as a list), and `associated_actors`.
I'm curious if anyone else has built similar tools, or if you've found a better way to handle this in your pipelines. I'm also wondering if I've missed any important fields that are particularly useful for automated enrichment.
Good luck when Mandiant changes their schema next month without notice. That's always the fun part with these vendor APIs. You'll get to rewrite your normalizer again.
Used their feed for about six months before we switched. The false positive rate on the actor associations was killing our analysts. Hope your playbooks filter for that.
CRM is a necessary evil
That's a fair point about schema changes. We've been on their feed for a few years now, and you learn to build a bit of resilience around it.
I think the key is abstracting the normalization logic from the data mapping. We treat our normalizer's output as an internal schema, then have a separate, lightweight mapping layer that translates the raw API response to that. When the feed changes, we only have to update that one mapping file. It's not zero work, but it's contained.
The false positive rate on actor associations is rough, I agree. We had to add a confidence threshold filter based on the number of supporting reports before an item even hits a playbook. Without something like that, it's just noise.
—Anita
Nice! Flattening those nested structures is a huge time-saver for playbook mapping. I took a quick look at your gist.
One thing I'd check is how you handle the list-of-dicts cleanup. It looks like you're using a simple list comprehension for `industries_targeted`, which is great, but if that field is ever missing or `null` in the raw JSON, you'll get a `TypeError`. A defensive get with an empty list as a default can save you some headaches during ingestion.
Also, for the `associated_actors`, are you pulling from the `actors` field or the `attributed_associations`? I've seen some variance there depending on the intelligence type. Might be worth a comment in the code so the next person on your team knows the source.
It's a solid start though. Sharing stuff like this is super helpful for the community 👍
Clean code is not an option, it's a sanity measure.
Abstraction and a mapping layer are the textbook answer. It's the work you're forced to do because the vendor provides a messy, unstable feed and calls it an API.
You're right that it's not zero work, but let's be clear about where the cost lands: on your team, for free, to clean up their product's output. They get paid for the data; you get paid to build the adapter. And their next schema change will still break something your mapping layer didn't anticipate because the field will just disappear.
The confidence filter is admitting the data is too noisy to use as-is. You're basically paying them to give you a problem, and then paying your analysts' time to solve it.
Trust but verify.
Oh, that initial normalization work is such a familiar pain point. I feel your relief getting something stable out of it. The moment you can reliably map those flattened fields to playbook triggers is a huge win for the team's velocity.
I'll toss in one hard-earned lesson from my own scars, though. That script becomes a single point of failure in your pipeline. When you're onboarding a new client or handing this off to a junior analyst six months from now, they won't remember the quirks that made you write it. We started adding a simple validation step right after the normalizer runs, checking for the presence of, say, at least three of our "mandatory" fields. If it fails, it logs the raw JSON that caused it. Saved us from silent ingestion failures more than once when a new, unexpected object type popped up in the feed.
Anyway, solid share. That kind of foundational glue work is what makes these integrations actually run day-to-day.
Implementation is 80% process, 20% tool.
I've been down that exact road. Flattening their nested objects into a consistent, playbook-friendly schema is the only way to make their data usable in a SOAR.
One nuance I'd add: for the `associated_actors` field, you might want to capture not just the actor name, but the type of association (like "attributed-to" or "leveraged-by") from the source object. We found that distinction mattered for our triage logic - a malware "leveraged by" an actor got a different priority than one "attributed to" them. Your flattening is the perfect place to embed that extra bit of context.
Also, seconding the defensive `get` for list fields. When their schema shifted and `aliases` was missing from a batch of malware objects, it blew up our pipeline mid-ingestion. A simple `.get('aliases', [])` saved a late-night page.