Vendor dashboards love showing me "threats blocked" and "alerts processed." That's activity, not security. I need KPIs that prove the SIEM is moving the needle on actual risk, not just generating busywork.
I'm looking for metrics that are:
* **Tied to outcomes, not inputs:** Mean Time to Detect/Respond/Remediate (MTTD/MTTR) is a start, but it's often gamed. How do you measure it *reliably*?
* **Actionable for tuning:** What's your process for measuring and reducing false positives in a way that doesn't increase risk? A simple ratio is useless without context.
* **Aligned with business risk:** How do you quantify coverage? Not just "we ingest 200 log sources," but "we have detection coverage for our critical Crown Jewels data flows." Is that even measurable?
Here's a basic query I might start with to track alert fatigue and tuning efficacy over time. This is pseudo-code for the general logic.
```sql
-- Weekly trend: Total alerts vs. actionable alerts
SELECT
DATE_TRUNC('week', alert_time) as week,
COUNT(*) as total_alerts,
COUNT(CASE WHEN triage_outcome = 'true_positive' THEN 1 END) as actionable_alerts,
(actionable_alerts / total_alerts) * 100 as signal_to_noise_ratio
FROM siem.alerts
WHERE alert_time > NOW() - INTERVAL '90 days'
GROUP BY week
ORDER BY week;
```
What specific KPIs and, more importantly, *methodologies* are you using to move beyond vanity metrics? I'm skeptical of anything that can't be reproduced in a similar query.
Show me the query.
That pseudo-query gets at a core operational metric, but you're right that the ratio alone is thin. For tuning efficacy, we track two things in parallel: the false positive rate and the 'coverage delta.'
Before any tuning, we document the detection logic's intended coverage, say, 'lateral movement via RDP.' After modifying to reduce noise, we run a retroactive query for the past 90 days with the *new* logic. If it misses any confirmed true positives that the old logic caught, that's a coverage gap. The goal is to reduce noise without creating those gaps. It turns tuning from a blunt exercise into a measured trade-off.
On your point about MTTD/MTTR being gamed, reliability comes from measuring from the adversary's action, not the alert. We use canary tokens or benign threat emulation tools to generate a discrete event chain, then measure the clock from that event to SIEM alert creation, and then to ticket closure. It's a controlled measurement, not based on noisy, real alert data.
Buyer beware, but with a spreadsheet.