Alright, buckle up. I usually haunt the cloud cost forums, but this one dragged me over because, surprise surprise, your SIEM is probably costing you a fortune *and* failing at its core job. I see this pattern all the time: teams splurge on massive data ingestion into their fancy SIEM, then the actual detection logic is so anemic it couldn't find a data exfil in an empty S3 bucket.
You're finding lateral movement clues manually? That means your SIEM is a glorified, overpriced log dumpster. The correlation engine is asleep at the wheel, and you're paying per GB for the privilege.
Let's break down why this happens, through my FinOps-tinted glasses:
* **Ingestion Overload, Analysis Under-spend:** You're likely ingesting everything from CloudTrail to VPC flow logs (which, by the way, are a cost nightmare if not filtered). But your detection rules are probably just simple, vendor-provided defaults looking for "impossible travel" or single-event "AdminLogin." Real lateral movement is a sequence. It's noisy. It looks like normal traffic until you connect 4-5 weird-but-plausible events across systems.
* **The Correlation Gap:** Your SIEM might be great at alerting on Event A and Event B, but what about the sequence of `unusual IAM call -> new security group rule -> spike in outbound traffic from a non-bastion host -> anomalous S3 `ListObjects` from that same host`? That's a story. Most out-of-the-box rules don't write novels.
* **Cost-Driven Blind Spots:** I'll bet someone saw the bill for ingesting process execution logs or full packet capture and said "Nope, too expensive." So you turned it off. Now you're missing the crucial `who -> what -> where` chain. You can't correlate a suspicious RDP login with `powershell.exe` spawning `net.exe` if you're not ingesting both auth logs *and* process logs.
Here's a crude but effective example. This is the *kind* of logic you need, expressed in a pseudo-rule. Your SIEM's syntax will differ, but the concept is key:
```sql
// Pseudo-Correlation Rule: Potential Credential Theft & Lateral Movement
// Looks for: Failed auth -> Successful auth from same source -> Rapid, successive network connections to multiple internal hosts
events = filter (
(event_type IN ("AuthenticationFailed", "AuthenticationSuccess") AND src_ip = "internal_range")
OR
(event_type = "NetworkConnection" AND dest_ip = "internal_range")
)
within timeframe = "10 minutes"
group by src_ip
trigger when for a grouped src_ip:
count(filter(event_type == "AuthenticationFailed")) >= 5
AND
any(event_type == "AuthenticationSuccess")
AND
count(distinct filter(event_type == "NetworkConnection", dest_ip)) >= 3
AND
the sequence of events matches: [Multiple Fails] -> [Success] -> [Multiple Connections]
```
You need to build these narratives. Start by mapping the manual clues you found back to raw log sources. What data did you need? Is it even being ingested? If not, you have a cost-vs-coverage talk to have.
Then, stop relying on singleton alerts. Build multi-event correlation rules that have a timeline. Use threat intelligence feeds (even internal ones, like a list of decommissioned servers) to enrich events. And for the love of all that is holy, **tune your baselines**. A developer SSH-ing between three boxes is normal. A finance server initiating SMB connections to five different departments in two minutes is not.
Your cloud bill is too high, and your security is too low. It's all connected.
Yeah, that makes a lot of sense. I've been trying to learn Splunk basics and the correlation stuff seems really hard to set up right. So the vendor rules are mostly just checking for one weird thing, but real attacks are a chain of "sorta weird" things?
If it's a sequence across systems, does that mean you have to build custom rules for your specific environment? Like, what a developer does on app-server-A versus database-B? That sounds like a massive project.