Hey everyone! I saw Panther's announcement about their new ML-driven detection rules. As someone still getting their head around basic SIEM stuff, I'm a bit skeptical.
It sounds cool, but I'm worried it might just be marketing fluff. Could someone who's tried it break it down for a newbie? Like, what does an "ML rule" actually look like in practice? Is it just a fancy threshold alert?
I'd love a simple example, maybe comparing a traditional rule to a new ML one. Thanks in advance to anyone who can explain! 😅
Fluff is generous. It's a rebranded statistical threshold.
Traditional rule: "Alert if failed logins > 10 in 5 minutes from same IP."
Their "ML" rule: "Alert if failed logins are 3 standard deviations above this IP's 30-day rolling average."
They just moved the baseline from a fixed number to a moving average. Still just counting events. They slap "ML" on it because the baseline auto-adjusts.
You're right to be skeptical. It's for the slide deck, not the SOC.
-- old school
Okay, I actually played with their beta. user292 isn't wrong about the rolling average example, but that's just the starter template. It gets more interesting.
They have a template for "impossible travel" that isn't just a fixed geodistance check. It builds a model of your regular login cities for each user, and the alert fires on a probability score, not just miles. So for a user who bounces between NYC and London weekly, a Tokyo login might not fire. But for a user who's only ever logged in from Denver, a London login would. That's a bit more than a moving threshold.
Is it earth-shattering? No. Is it fluff? Not entirely. It's a decent step toward detections that adapt to your org's actual behavior instead of a generic rule. The real test is if they let you tune the models or if it's just a black box.
Try everything, keep what works.
Probability scoring for location is the interesting part, if it's real. The devil is in what they consider a "regular" city and the training window.
Seen similar attempts in other platforms where the model gets confused by VPNs or becomes useless for new hires. If you can't see or adjust the confidence threshold, it's just a slower, more opaque threshold alert.
I'd want to know what happens when their model decides a contractor's once-a-year conference city is an anomaly. Does it fire every day they're there?
YMMV