Hey everyone, I've been deep in the weeds with our Firebox T-series trying to get per-user bandwidth monitoring working without buying the extra licensed features like Dimension or the full Threat Detection and Response suite. Our budget is tight, but we really need to see which users are hitting our WAN links hardest, especially for non-standard protocols that our basic reports don't break out.
I've built a workaround using the Firebox's built-in tools and a few external integrations. The core idea is to use the Firebox's **Syslog** output, parse it externally, and then correlate IP addresses to users. It's not real-time glamour, but it gives us the daily/weekly trends we need.
Here's the basic flow:
* Configure the Firebox to send detailed traffic logs to a syslog server. You'll want to ensure the log format includes source IP, bytes sent/received, and destination port.
* Run a syslog daemon on a local Linux VM or container (rsyslog works great). This collects the raw data.
* Use a script (I used Python) to parse the logs, summarize bytes per source IP per time period, and then—this is the crucial part—tie those IPs to specific users.
The user/IP mapping was our biggest hurdle. We don't have an always-on DHCP lease list to query. Our solution combines two data sources:
1. **Firebox DHCP lease logs:** Also sent via syslog. We parse these to get a timeline of which IP was assigned to which MAC address.
2. **Active Directory logon events:** A small agent on our DC forwards relevant events (user, hostname, IP, timestamp) to our parsing script.
The script then correlates the three data streams: traffic logs (IP, bytes), DHCP logs (IP, MAC, timestamp), and AD logs (hostname, user, IP, timestamp). The result is a summarized report of bandwidth per user.
Here's a super simplified snippet of the correlation logic in Python:
```python
# Pseudo-structure of our data dictionaries
traffic_logs = {'10.0.1.45': {'timestamp': '2023-10-01 14:30:00', 'bytes_sent': 1500000}}
dhcp_leases = {'10.0.1.45': {'mac': 'aa:bb:cc:dd:ee:ff', 'hostname': 'jdoe-laptop', 'lease_start': '2023-10-01 14:15:00'}}
ad_logons = {'jdoe-laptop': {'user': 'jane.doe', 'logon_time': '2023-10-01 14:20:00'}}
# Correlation loop (conceptual)
for ip, traffic in traffic_logs.items():
if ip in dhcp_leases:
hostname = dhcp_leases[ip]['hostname']
if hostname in ad_logons:
user = ad_logons[hostname]['user']
# Accumulate bytes for this user
user_bandwidth[user] += traffic['bytes_sent']
```
We output this to a simple dashboard using Grafana (fed by a SQLite database). The setup requires some maintenance, but it cost us nothing but time.
Has anyone else tackled this? I'm curious if there's a more elegant way to pull user identity from the Firebox itself, perhaps via its LDAP query capabilities, to simplify the pipeline. I tried using the Firebox as a RADIUS client for accounting, but the data wasn't as granular as I needed for per-application insights.
api first
api first
The syslog to script pipeline is solid for a DIY fix. That user/IP correlation step is where most people trip up though, especially with DHCP.
If you're already running AD or have a decent directory, you can pull DHCP lease logs to map IP to hostname, then cross-reference with directory auth logs for the user. It's messy, but it works.
Just watch your log volume. Parsing raw traffic logs for an entire WAN can bloat fast. You'll need a rotation and aggregation script running daily or you'll drown in data.
cost optimization, not cost cutting
That's the part that seems tricky. You mention cross-referencing with directory auth logs. How do you handle users who are logged in but not actively generating traffic at the exact time a log entry is written? Isn't there a timing gap that could misattribute bandwidth?
Yeah, the mapping part is the real hurdle. I've done something similar with our Palo Alto logs feeding into a Grafana stack.
Instead of trying to match logs in real-time, I set up a separate process that snapshots the DHCP lease state and AD logged-in users every 5 minutes, dumping those relationships into a small PostgreSQL table. The traffic log aggregation script then does a "closest in time" lookup for each IP in that table. It's not perfect, but it gets you about 95% accuracy for daily totals, which is usually enough to spot the big bandwidth users.
Just be ready for some noise from shared devices or meeting room PCs.
cost first, then scale
That 5-minute snapshot approach is clever, it deals with the problem of log timestamps being slightly out of sync with the actual user-to-IP binding moment. I've done something very similar, but used a Prometheus/StatsD combo to scrape active DHCP leases and push them as a gauge.
The 95% accuracy you mention is spot on. That last 5% of weirdness almost always comes from VPN clients, where the internal IP is short-lived or the user mapping happens at a different layer altogether. For those, I had to add a separate log ingest from the VPN concentrator itself.
What do you do about wireless clients that might roam between APs and get a new IP? We saw that throw off our 'closest in time' lookup until we tightened the DHCP lease time.
pipeline all the things
Exactly, that "closest in time" lookup in Postgres is the pragmatic way to handle the lag. I'm glad you mentioned the 95% accuracy, because that's the realistic target folks should aim for.
One nuance I've run into with this method is handling users who appear in multiple snapshots under different IPs within a short window, like someone hopping from wired to wireless. My aggregation script had to get smarter about not double-counting their traffic if the IP change happened mid-session. I ended up adding a simple deduplication pass that checks for the same user ID across adjacent IPs within a 10-minute window.
The shared device noise is real, too. We eventually tagged those specific machine hostnames in a lookup table so the final report could flag that traffic separately. It stopped a lot of "why is the conference room using so much bandwidth?" emails.
Spreadsheets > opinions
I've gone down this exact syslog road with our FortiGates, and that user/IP mapping step is the real grind. Your setup is spot on for a tight budget.
One thing we learned the hard way: log volume. You mentioned parsing for trends, but the raw traffic logs from a busy WAN can absolutely bury a syslog VM if you're not careful. We had to add an intermediate step where a lightweight cron job strips out only the fields we need (timestamp, src IP, bytes) and discards the rest every hour. Saved our storage and parsing time.
Also, for the mapping, if you have any sort of endpoint management tool (like Lansweeper or even a simple scheduled `netstat -an` script on a DC), you can scrape logged-on users periodically. It's another layer of data, but it helps fill in the gaps when DHCP logs are too sparse.
Right-size everything
Yeah, the log volume warning is huge. We tried this with a WatchGuard and filled up a 50GB disk in a weekend. That intermediate cron job idea is smart.
When you say you scrape logged-on users from a DC, do you mean querying active sessions against the server itself, or polling each workstation? I'm trying to picture how to set that up without disrupting users.
Hey, thanks for laying out that flow so clearly. The user/IP mapping is indeed the part where most DIY setups either sink or swim. Your point about needing to see non-standard protocols is key, something the basic vendor reports often just gloss over.
I've seen a few teams get tripped up on that final mapping step when they rely solely on DHCP logs, especially if there's a lot of device churn or short leases. It might be worth adding a second data source early on, like a periodic query to your directory service for active user sessions, just to have something to cross-check against. It adds a bit more complexity to the script, but it can save you from those "who was at this IP at 2 AM?" mysteries later.
Solid approach overall. I'm curious, have you hit any snags with the log volume or format from the Firebox itself? Sometimes the default syslog output can be surprisingly verbose for traffic logging.
Let's keep it real.
Deduplicating across adjacent IPs is the right move. That 10-minute window is critical.
We had to narrow ours to 5 minutes because of our DHCP lease times. Even then, we still see double-counts during certain VPN failover events where the user session persists but the internal IP flips instantly. Our report now tags any traffic where a user had >1 IP within that window for manual review.
The shared device lookup table is mandatory. We also added a weighting factor to split the traffic 50/50 if two users were logged to a shared host concurrently, based on the session start times in the snapshot. It's crude but better than assigning it all to the last user.
Five nines? Prove it.