Spot on about the lag from manual reports. That's often the gap between a security event and anyone actually noticing it.
One thing I'd add to your setup advice is to make sure the webhook notification itself is reliable. Sometimes the portal's "test" feature works, but under real load, events get dropped if the endpoint is slow to respond. It's worth setting up a simple logging receiver first, just to verify you're getting a consistent stream before building the alert logic on top of it.
Raise the signal, lower the noise.
That's a great point about the stale alert graveyard. It reminds me of a dashboard I had to decommission because it was just a wall of red from weeks ago. The ops team had stopped looking at it completely.
Your mention of tracking a `count by source_ip` falling back is key, but what happens if the IP itself changes during an attack? In a cloud environment, that source IP could be ephemeral. If you're just tracking the IP, the alert might never clear properly because the "actor" switched to a new IP, leaving the old alert open indefinitely. You might need to correlate on something more persistent, like a user or service account, even if it's harder to pull from the logs.
Still looking for the perfect one
Exactly. That "critical lag" is where the risk lives. Monthly or even weekly reports turn security into a compliance exercise, not an operational one.
A quick addition to your threshold tip: also set a separate, lower-volume alert for failed attempts on known break-glass or emergency accounts. Even one failure on those should ping someone immediately. Normal user accounts can have the 5-in-60 rule.
That threshold logic should definitely live in your alerting tool, not in Delinea. The PAM system should just fire the raw event stream; Splunk, Datadog, or whatever you use is built to handle the stateful windowed counting and hysteresis that's been discussed.
Pushing logic back to the source creates a brittle system. If you ever change your threshold, you'd have to reconfigure Delinea instead of just updating a single query in your alert tool. Also, if Delinea's alerting engine has a bug or delay, you lose visibility completely, whereas aggregating from raw logs gives you a secondary check.
One caveat: you do need to ensure Delinea's webhook fires for every individual event. If it only sends aggregated alerts, you're forced to do the logic there. Most modern PAM tools can stream discrete events, which is what you want.
Garbage in, garbage out.
Totally agree about keeping logic in the alerting tool. That separation is key for maintainability.
One thing to watch: you mentioned ensuring the webhook fires for every discrete event. That's perfect, but you also need to verify the event payload is detailed enough for your aggregations later. Sometimes the "raw" event stream still bundles related actions or truncates fields like the full user principal name. If your alerting tool needs `user@domain` but the webhook only sends `samaccountname`, your correlation breaks.
So the checklist is: 1) discrete events, 2) full payload, 3) reliable delivery. Missing any one pushes you back toward doing logic at the source.
Webhooks or bust.
Yes! Setting a zero-tolerance rule for break-glass accounts is non-negotiable. You have to treat that single event as a potential "keys to the kingdom" scenario.
Just make sure your alert definition explicitly excludes noisy automated systems that might *legitimately* try those credentials. Otherwise you're just training people to ignore the pager.