Hey everyone! Still pretty new to this side of things, but I've been trying to follow Mandiant's daily intel digests. I found myself getting overwhelmed trying to spot just the *new* TTPs each day in the reports.
So I wrote a small Python script to help. It basically compares the current report to yesterday's and prints out only the new tactics and techniques. It's super basic and just uses some simple string matching for now, but it really helped me cut through the noise.
Would love any feedback if you have a sec! I'm sure there are better ways to do this, maybe with their actual API? This is just my first attempt at parsing this kind of data. Thanks! 😊
That's a smart approach for cutting through the repetition in daily feeds. String matching gets you surprisingly far, but you're right to think about their API. Mandiant's schema is JSON-LD, which is structured for exactly this kind of programmatic consumption. A simple diff on the raw JSON objects, keyed on the technique ID, would be more reliable than string matching on the rendered text. It'd prevent missing a technique just because the description wording changed slightly between days.
I'd also suggest building a small local cache, even just a SQLite table, to store the technique IDs and first-seen dates from your runs. That way you can track trends over a week or month, not just day-to-day. You'll start seeing which actors are reusing techniques versus trying new things.
Good instinct to move off text scraping. The API is indeed JSON-LD.
Your diff approach is solid. Key it on the technique ID, not the description. Cache the IDs locally to see what's genuinely new over a week, not just since yesterday. You'll miss techniques that were added then removed if you only look day-to-day.
For a next step, hook it into a simple dashboard. Flag techniques seen for the first time in the last 7 days. That's the signal you want.
Trust, but verify
Agreed on keying the diff on technique ID. That's the only stable reference in their schema. The JSON-LD has a `@id` field for each technique that's perfect for this.
The weekly dashboard idea is good, but you need to consider the data volume. If you're pulling all daily reports, you'll have a huge list of IDs after a few months. A rolling cache with a TTL is better than indefinite storage. A simple script can prune entries older than your analysis window, like 30 or 90 days.
You could also structure the cache to include the reporting actor or malware family alongside the technique ID and first-seen date. That lets you build a second view showing which actors are introducing the most new techniques in your time window, which is often more actionable than just a raw list of IDs.
- Mike
Good first step, string matching can get you 80% of the way there. The API route others mentioned is definitely better, but I'd stick with your diff logic and just change the source.
Instead of parsing the text report, fetch the structured data. Their feeds are often just static files. You can wget the JSON-LD and run your diff against that. It's one extra line in your script and you won't get burned by a typo in the human-readable summary.
Cloud costs are not destiny.