Everyone's talking about detection coverage, but most teams are just checking boxes. They deploy a SIEM, turn on the default rules, and call it a day. That's a compliance exercise, not security. How do you know your Elastic Security deployment actually sees the attacks that matter to your environment?
I use Atomic Red Team tests to simulate specific adversary techniques. It's a straightforward way to validate if your detection rules fire and if your alerts contain the necessary context. Here's my blunt walkthrough.
First, I map a relevant MITRE ATT&CK technique to my environment. For example, T1059.001 (Command and Scripting Interpreter: PowerShell). I then run the corresponding Atomic test, which is often a simple one-liner executed from a test machine.
* **The Critical Step:** I don't just look for an alert in Elastic. I scrutinize the alert's fidelity.
* Does it correctly identify the process lineage, user, and host?
* Does the rule logic catch the variation of the technique, or is it too brittle?
* Is the alert noisy? Would it fire on legitimate admin activity?
I've found default Elastic rules often need tuning. They might be too broad, generating alerts for benign PowerShell use by our sysadmins, or too specific, missing slight variations of the same technique. The goal is to close the gap between what the simulation does and what your rules actually catch.
This process also exposes blind spots in data sourcing. A test might succeed because the necessary ECS field isn't being populated by your endpoint agent or network sensor. You learn real quick if you're actually collecting the right data or just assuming you are. Start with techniques that are high-probability for your industry and work from there. Trust, but verify.
—Daniel
Trust but verify.
Totally agree. That final step of checking the alert's context is where most validation efforts fail. It's not just about the "alert fired" event.
I run similar tests as part of our CI pipeline, triggered on detection rule updates. The automation confirms the detection, but a human has to review the resulting alert for the exact details you mentioned - like proper parent process tagging. If the alert just says "powershell.exe spawned cmd.exe" without the user and source host, it's useless for a real response.
Your point about brittle logic is key too. I've seen rules that only catch the exact Atomic Red Team command syntax. A slight variation in arguments or using a different built-in module and the rule goes silent. Tuning for that without creating noise is the real art.
Keep automating!