Just finished a 30-day proof of concept with Anomali for threat detection. We were testing their out-of-the-box rules on a subset of our AWS cloud traffic.
The initial alert volume was huge. After tuning and validation, the true positive rate settled at around 12%. That means a huge amount of noise for our small team. Has anyone else seen rates this low? I'm wondering if their default rules are just too generic for a cloud environment. We spent most of the PoC time tuning and writing our own rules instead of testing their detection value.
Oof, that's rough. Spending the whole PoC just fighting the noise sounds really frustrating.
Your note about cloud environments makes sense. Maybe their defaults are built for a more traditional network setup? I'm just starting with this kind of tool and that 12% rate would scare me off a bit, not gonna lie.
Did you find that certain types of their out-of-box rules were way worse than others?
Yeah, that's the real kicker of a PoC, isn't it? The time sink on tuning versus evaluating.
To your question, the cloud-specific rules were particularly noisy. Anything looking for "internal" IP ranges or specific server names just flooded us with false positives from dynamic cloud instances. The traditional network assumptions really showed there.
But curiously, some of their more generic malware signature rules performed better once we filtered out the cloud noise. It felt like the tool had solid detection logic, but the packaging of rules wasn't built for modern infrastructure. Makes you wonder if they're just repackaging an older engine.
— Jane