Having recently migrated a mid-sized deployment from a commercial UTM appliance to an OPNsense firewall, I immediately encountered a significant observability gap: the lack of detailed, historical per-client bandwidth usage out of the box. The built-in Traffic Graphs are useful for aggregate interface views but fall short for answering questions like "which department's VLAN is saturating the WAN link at 2 PM daily?" or "is that new IoT device actually phoning home excessively?" The official solution, the `bandwidthd` or `nTopng` plugins, are excellent but represent a recurring cost for the commercial versions or require managing the open-source builds.
Naturally, my inclination was to avoid additional licensing fees and explore what could be accomplished with the underlying FreeBSD system and existing OPNsense/pfSense components. The core requirement is to capture IP-level throughput data, associate it with client identifiers (hostname, DHCP lease info), and persist it for trend analysis. The built-in firewall logs (`/var/log/filter.log`) contain this data but in a high-volume, non-aggregated form.
After several iterations, I've developed a workflow that provides satisfactory per-client monitoring without purchasing a plugin. It hinges on three core components:
1. **NetFlow Data Export:** Leverage the built-in `softflowd` package to export NetFlow v5 data from the firewall interfaces.
2. **Local Collection & Aggregation:** Use a lightweight NetFlow collector (`nfdump` or `flow-tools`) running directly on the firewall to capture and aggregate the data.
3. **Data Presentation:** Periodically generate reports from the aggregated flow data and optionally expose them via a simple web interface.
Here is a concise configuration outline for OPNsense (pfSense steps are analogous, with package names like `softflowd` vs `softflowd-pfSense`):
**Step 1: Enable NetFlow Export via softflowd**
Install the `os-softflowd` plugin via the GUI. The critical configuration is to set the destination to `127.0.0.1:9995` (for local collection) and select the interfaces to monitor.
**Step 2: Install and Configure a NetFlow Collector**
SSH into the firewall and install `nfdump` from the FreeBSD packages.
```bash
pkg update
pkg install nfdump
```
Create a simple startup script to run the collector. A basic `rc.conf` entry or a service file can be used. The collector command would be:
```bash
nfdump -D -l /var/netflow/log -p 9995 -S 1 -z -w /var/netflow/capfile
```
This runs `nfdump` in daemon mode (`-D`), listening on port 9995, rotating files daily (`-S 1`), and compressing old data (`-z`).
**Step 3: Generate Readable Reports**
A cron job can be set up to periodically process the binary flow data into human-readable summaries, keyed by client IP. A simple script can leverage `nfdump`'s filtering and aggregation capabilities:
```bash
nfdump -R /var/netflow/capfile -s srcip/bytes -n 20 -o csv > /var/netflow/daily_top_clients.csv
```
To add client hostnames, you can parse the CSV and cross-reference with the DHCP lease file (`/var/dhcpd/var/db/dhcpd.leases`) or the firewall's static ARP table.
The primary trade-offs of this approach are:
* **Storage:** Flow data can be voluminous. Aggregation and log rotation are mandatory. A dedicated dataset or external storage mount for `/var/netflow` is advisable.
* **Processing Overhead:** `softflowd` and `nfdump` add CPU load, which is negligible on modern hardware for sub-gigabit speeds but should be monitored.
* **Real-time Latency:** This method is best for historical trend analysis (e.g., daily reports). It is not a real-time traffic graph. For near-real-time views, you could pipe data to a local `InfluxDB` instance and use `Grafana`, but that increases complexity significantly.
This setup provides a cost-free, extensible foundation for per-client bandwidth accountability. It requires comfort with the command line and cron, but the data fidelity is high. I'm curious if others have pursued similar self-built monitoring stacks and what optimizations or alternative data pipelines you might have implemented.
- alex
Measure twice, cut once.
So you're building on the built-in firewall logs. Are you planning to parse the filter.log directly, or did you find a way to get netflow data out of the firewall itself first?
Right on the money - I'm pulling from the filter.log directly. It felt like the most direct path, and it's already there logging every passed/blocked packet. The trick is the sheer volume; a busy firewall generates a monstrous log file, so you really need a script to parse it in real-time or batch-process it on a schedule.
I've been using a Python script that runs via cron every 5 minutes, sums up the bytes for each internal IP, and shoves the totals into a local database. The caveat, and it's a big one, is that this only measures traffic that the firewall rules *allow*. If you're logging blocked traffic too, you could add that in, but for my use case of "who's using the WAN," the passed traffic is what counts.
Have you tried the netflow route? I looked into it, but configuring netflow export on OPNsense felt like more work than just parsing the logs I already had enabled.
Pipeline is king.