Hey folks, been using Mandiant's Threat Intel for about six months now, primarily for vulnerability prioritization in our Salesforce and connected marketing cloud environments. The predictive scoring and threat context are great for cutting through the noise of a massive CVE list.
But I'm hitting a wall when it comes to internal validation. We've been patching based on their high and critical "likely to be exploited" flags for a while. Has anyone done a formal, retrospective analysis to see if those predictions actually matched your real-world exploitation attempts *after* patches were applied? I'm talking about correlating your own incident data over, say, a 12-month period against the predictions you acted on.
I'm trying to build a business case for continued investment, and our RevOps/analytics team is asking for harder ROI. We can track "patches applied," but proving "breaches prevented" because we trusted a specific prediction is trickier. Would love to hear if anyone has set up a feedback loop or metrics framework for this.
Our current process is basically: ingest feed -> prioritize based on Mandiant score + internal asset criticality -> patch. The gap is in closing that loop to see if the intel was truly predictive for *our* infrastructure, not just globally accurate. Any experiences or workarounds shared would be awesome.
Data is the new oil, but it's messy.
That's a really interesting point about the validation gap. I haven't done a formal retrospective analysis myself, but I work in reporting and have had similar conversations with our security team.
One approach we discussed was trying to measure the inverse: for incidents that did occur, could you trace the root vulnerability back and check if it was ever flagged by your predictive source? If you find exploited vulnerabilities that were never scored as high-priority, that's a strong data point on the model's potential blind spots.
It seems like you'd need a pretty mature logging and incident taxonomy to even attempt that correlation. Has your team considered looking at it from that reverse angle?