Skip to content
Notifications
Clear all

How do I track whether using this actually improved our MTTD?

2 Posts
2 Users
0 Reactions
3 Views
(@cloud_ops_amy)
Estimable Member
Joined: 5 months ago
Posts: 128
Topic starter   [#15630]

Hey everyone, I've been piloting Mandiant Threat Intelligence for my team over the last quarter. The intel feeds themselves are great—high signal, good context—but I'm hitting a wall when trying to justify the ongoing cost to leadership. The classic question came up: "How do we know this is actually making us faster at detecting threats?"

We've got our Mean Time to Detect (MTTD) baseline from before we onboarded, but I'm struggling to attribute improvements directly to the intel. A lot of other factors play in: alert tuning, new detection rules, analyst training.

Here’s my current approach to measurement:

* **Tagging Alerts:** We've configured our SIEM to tag any alert where the IoC or TTP context came directly from a Mandiant feed.
* **Detection Timeline Tracking:** For any incident, we're trying to document the point where the Mandiant-sourced data first entered our workflow.
* **Threat Hunts:** Measuring the time it takes to investigate a hunt hypothesis with and without the enriched context from their portal.

The part I'm wrestling with is creating a clean, automated metric. Has anyone built a dashboard or a process that effectively isolates the intel's impact?

For example, I'm thinking of a simple Lambda function that logs when a Mandiant-sourced IoC matches in our environment, timestamps it, and compares it to when our own internal detection would have theoretically fired. But that "theoretical" time is the fuzzy part.

```python
# Pseudo-code for the idea
def evaluate_intel_impact(event):
# event contains matched Mandiant IoC and timestamp
mandiant_detection_time = event['time']

# Query for similar events without Mandiant intel (historical baseline)
baseline_detection_time = get_baseline_detection(event['ioc_type'])

time_delta = baseline_detection_time - mandiant_detection_time
log_metric("MTTD_Reduction_Seconds", time_delta)
```

Would love to hear how others are tackling this. Are you comparing to industry baselines? Focusing more on the quality (false positives reduced) rather than pure speed? How do you track the operationalization of a specific TTP report from their portal into a new detection rule, and then measure *that* rule's efficacy?

-- Amy


Cloud cost nerd. No, I don't use Reserved Instances.


   
Quote
(@grafana_knight_shift_2)
Estimable Member
Joined: 2 months ago
Posts: 110
 

Tagging alerts is a solid start, but for attribution you need to tie that tag to the *detection moment*. We built a Prometheus counter that increments when an alert fires with that intel tag *and* it's the first alert in a new incident group. That gives you a numerator: "detections directly sourced from this intel."

The real trick is the denominator. You need a way to estimate how long those same threats would have taken to find without the intel. That's the attribution hole. We use a manual, quarterly review of a sample of those intel-sourced incidents. Analysts answer a simple question: "Ignoring the automated alert, when would our next investigative pivot have found this?" It's subjective, but the aggregated estimate creates a comparison point for your MTTD calculation. It's not fully automated, but it passes the leadership sniff test.


Sleep is for the weak


   
ReplyQuote