Hey folks, new to the Anomali platform and I'm trying to set up a realistic proof-of-concept in our lab. The sales demo was slick, but I'm the one who'll be building the dashboards and tuning the alerts, so I need to get my hands dirty.
We're a relatively small shop. Our central syslog-ng server only holds about 80GB of aggregated logs (firewall, app servers, auth) over a 30-day retention period. I've read the docs that talk about "big data" and training periods, which has me worried. My manager's question was simple: "Can we even get a meaningful security signal from such a small dataset, or are we just paying for a fancy log search UI?"
My main goals for the PoC:
* See if it can baseline our normal SSH/AD authentication patterns and flag real outliers.
* Test the built-in threat intelligence matching against our perimeter firewall denies.
* Understand if the correlation rules would have any enough data to work with.
From my Grafana/Prometheus world, I know you can do useful anomaly detection on a single metric stream with just a few weeks of data. But Anomali seems to be a different beast. Has anyone here run a successful evaluation with a similarly modest log volume? Were you able to generate any high-fidelity alerts, or was it mostly just noise?
I'm planning to feed it a straight syslog TCP stream. Any pitfalls in the parser setup I should watch for? I'm used to writing Grok patterns for Logstash, but their parser seems... different.
Appreciate any night-owl insights from those who've been in the trenches with it.
--- hoot
Silence is golden, but only if you have alerts.
80GB over 30 days is far more than enough for the use cases you listed. The "big data" talk is mostly marketing fluff for the enterprise slide deck.
Your manager's question is the right one. The meaningful signal depends more on the *variety* and structure of your logs than the raw volume. Can syslog-ng parse them into distinct fields (source IP, user, event ID)? If it's just a giant blob of text, no amount of terabytes will help.
I've seen decent baselines for auth patterns built from two weeks of clean data. The real test is whether the platform's "outliers" are actually weird events you missed, or just Friday afternoon logins from the IT guy's home IP. Tune it to ignore that, and see if it then catches something real. That's your PoC.
Tom W.