I've sat through one too many "next-gen" security dashboard presentations that look like the bridge of the USS Enterprise, cost six figures a year in licensing, and yet somehow make it harder to spot a simple east-west probe than a spreadsheet from 2003. Everyone's obsessed with "anomaly detection" and "AI-driven threat graphs," forgetting that most lateral movement follows a painfully predictable pattern: once they're in, they're going to start knocking on doors between subnets.
So, I threw the kitchen sink of vendor bloat out the window and built a visualization in Kibana that actually shows my team what they need to know: what's trying to talk to what, where it shouldn't be, and how noisy it is. The goal is simple: visualize attempted connections *across* our critical subnet boundaries, focusing on failures (which are often probes) and unexpected successes.
The core of it is two custom visualizations built on a transform of our network flow logs (AWS VPC Flow Logs in this case, ingested into Elastic). The magic is in the aggregation, not in fancy machine learning.
First, I created a scripted field to categorize the traffic. We care about "Cross-Subnet" traffic, defined as source and destination IPs not being within the same major subnet (e.g., 10.0.1.0/24 to 10.0.2.0/24). The logic is straightforward:
```json
{
"script": {
"source": """
def src_ip = doc['srcaddr'].value;
def dst_ip = doc['dstaddr'].value;
// Simple function to get the /24 network (adjust for your schema)
def getNet(ip) {
String[] octets = ip.splitOnToken('.');
return octets[0] + '.' + octets[1] + '.' + octets[2] + '.0';
}
if (getNet(src_ip) != getNet(dst_ip)) {
return "Cross-Subnet";
} else {
return "Internal";
}
""",
"lang": "painless"
}
}
```
With that, the main dashboard has two key components:
1. **A weighted subnet relationship matrix:** This is a modified Tile Map visualization. Each tile represents a source-destination subnet pair (e.g., "Prod-Frontend -> Prod-Database"). The size of the tile is weighted by the **count of rejected connection attempts** (flow log action = 'REJECT'). A large tile immediately tells you where someone or something is persistently trying to breach a boundary. Color can denote the ratio of rejects to accepts.
2. **A temporal heat map of cross-subnet traffic:** This is a standard Heat Map visualization, but filtered on our "Cross-Subnet" scripted field.
* Y-axis: Source subnet.
* X-axis: Time (bucketed by hour).
* Color intensity: Count of distinct destination IPs contacted.
A vertical stripe of intense color shows a single subnet scanning everything at a specific time. A horizontal line shows a single source hitting multiple subnets consistently (maybe a misconfigured service account).
The beauty is in the simplicity and cost. This runs on:
* The flow logs you're already (hopefully) collecting.
* Elastic's built-in visualizations.
* A few hours of scripting and dashboard building.
No "threat intelligence" subscription, no external graph database, no seven-figure SOAR platform. It surfaces the obvious stuff immediately, which is 95% of what my ops team needs to triage. The other 5%? That's what actual investigation is for. This just gets you to the right starting point without the vendor-induced coma.
keep it simple
This is exactly why I started looking at our own VPC Flow Logs. The fancy tools we tried were drowning us in alerts for weird DNS lookups but would miss a steady scan across our app tier subnets.
I'm really curious about your scripted field logic. Did you have to account for things like legitimate traffic from your bastion hosts, or do you just filter that out post-aggregation?
We're still small enough that visualizing this manually in Kibana feels doable, but I worry about scaling the noise.
Couldn't agree more about the "bridge of the USS Enterprise" effect 😅 It's all flash, no situational awareness. I've seen teams miss a slow subnet crawl because the "AI threat score" was busy flagging backup traffic.
What really sold me on a simple approach like yours was hooking it to PagerDuty. We set a threshold on those cross-subnet failures, and the alert that fired was so clear - "src: dev subnet, dst: prod database, 40 RSTs in 5 min". No deciphering needed.
How are you handling the time window? Do you track a rolling baseline for "normal" cross-talk, or is it more of a static allow-list you filter out?
If it's not monitored, it's broken.
You're already ahead by starting with VPC Flow Logs. The raw data doesn't lie, which is more than I can say for most "curated" threat feeds.
>scripted field logic
My advice: don't do it in Kibana. Perform the enrichment and tagging *before* ingestion. Use a Lambda or a small log processor to stamp each flow log entry with a "cross_boundary: true/false" tag based on your subnet CIDR lists, and explicitly tag known bastion/jumpbox IPs as "source:trusted_jump". Doing it in a scripted field will cripple your dashboard performance at any real scale. Filtering post-aggregation is a recipe for missing context when you're investigating.
Noise scales with entropy. If you're worried about scaling now, you've already outgrown a static allow-list. You'll need a rolling baseline of connection counts per src/dst subnet pair, but the critical part is alerting on the *change* in failure rates (SYN/RST), not just the volume. A steady scan increases failures, not just attempts.
- Nina
Totally get the DNS alert fatigue, that's a classic! We had the same thing with our cloud vendor's "threat detection" module.
For the bastion host issue, I filter it at two points. The Kibana viz has a static filter to strip out known jumpbox IPs, but more importantly, we tag that traffic at the log shipper level (using Fluentd). That way, even raw searches are cleaner.
On scaling the noise, you're right to worry. A static allow-list gets unwieldy fast. What helped us was adding a simple baseline: the viz calculates the 7-day rolling average of unique destination ports per source IP per subnet pair. A spike above that baseline is what gets our attention, not just the raw connection count. It catches those slow crawls beautifully.
Happy reviewing!
>scripted field to categorize the traffic
That's going to break on you when the dataset grows. Enrich at ingest, not query time. Use a Lambda to tag each flow log entry with `cross_boundary: true/false` based on a CIDR lookup table. Your Kibana viz should just filter on that pre-computed tag.
Focusing on failures is smart, but don't ignore successful connections that are just as anomalous. A successful cross-subnet call to a database port from a web-tier instance is the real payload.
Infrastructure is code.