Hey everyone! 👋 I've been knee-deep in evaluating our edge firewall options for a new colo deployment, and I know the perennial pfSense vs. OPNsense debate comes up a lot here. Rather than just going by specs or gut feel, I decided to run a structured set of iperf3 throughput tests while monitoring CPU utilization. My goal was to see how each handled sustained traffic under identical hardware, focusing on the 10 Gbps capability we need.
**My test rig:**
- Hardware: Protectli VP4670 (Intel J6412, 4x Intel i226-V 2.5GbE ports, 16GB RAM)
- Hypervisor: Proxmox 8.1 (allocated 4 cores, 8GB RAM to each firewall VM)
- Firewall Versions: pfSense Plus 23.09.1 & OPNsense 23.7.11 (both clean installs)
- Test Client/Server: Two Linux boxes with 10GbE NICs, connected to separate firewall ports.
- Test methodology: Ran iperf3 in TCP mode with parallel streams (2, 4, 8, 16), 60-second duration each, measuring average throughput and observing `top` for firewall CPU use.
Here are the raw results, running with 8 parallel streams (which gave the most consistent max throughput):
| Firewall | Avg Throughput | CPU Utilization (system+user) | Notes |
| :--- | :--- | :--- | :--- |
| **pfSense Plus** | ~2.38 Gbps | ~87% | CPU cores clearly saturated |
| **OPNsense** | ~2.41 Gbps | ~91% | Similar saturation, slightly higher system % |
Both essentially maxed out the CPU on this hardware at just under 2.5 Gbps. Digging deeper, I noticed a key difference in how the network stacks handled the load.
**Key observations:**
* **Driver & Stack Behavior:** OPNsense (using the FreeBSD 13.2 base) showed a slightly higher system CPU percentage, which I suspect is due to its more recent `netmap` integration and different network stack tuning out-of-the-box. pfSense (FreeBSD 14) had a marginally lower system load but similar total saturation.
* **Tuning Attempts:** I tried enabling `hw.ibxn.max_rss` for RSS support on the i226-V interfaces and played with `net.isr` bindings. This gave a **~5-10% boost** to both, but the ceiling was still the CPU. The tuning felt more straightforward on pfSense due to its longer history with these drivers.
* **Monitoring During Test:** The built-in graphs (Netgraph on pfSense, Flowd on OPNsense) were both helpful, but OPNsense's real-time traffic graph updated more smoothly under load. However, for pure throughput, the difference was negligible.
**My takeaway for 2.5G+ deployments:**
If you're aiming for sustained multi-gigabit throughput on similar low-power Celeron hardware, **both will hit a CPU wall before hitting line rate on 2.5GbE ports**. You'll need more powerful cores (think Xeon or Ryzen) or hardware offloading (which these NICs don't fully support). The performance difference between the two firewalls, in my testing, was statistically insignificant.
Where they *diverge* is in the operational feel during high load:
- **pfSense** felt a tad more stable in the UI when CPU was pegged.
- **OPNsense** provided more detailed real-time graphs out of the box.
For our use case, we're leaning towards OPNsense because of its integrated Zenarmor and better observability into per-rule logging during incidents—but purely on throughput, it's a tie.
Has anyone else done similar stress tests on Atom/Celeron boxes? I'm curious if you've found tuning knobs that made a bigger difference than the ones I tried.
—jr
—jr