Having recently migrated my home lab's edge firewall from a commercial appliance to OPNsense, I undertook a structured performance analysis under conditions matching a typical 1 Gbps symmetrical fiber deployment. My primary metrics were throughput under various packet sizes, latency under load, and system resource utilization, with a focus on data pipeline reliability for my internal analytics workloads.
**Test Hardware & Configuration:**
* **Platform:** Protectli VP2420 (Intel J6412 Quad-core @ 2.0GHz, 16GB DDR4 RAM, 256GB NVMe)
* **NICs:** 4x Intel i225-V 2.5GbE ports
* **OPNsense Version:** 24.1.9_1 (Hardened)
* **Key Packages:** `os-sensei` (for application layer insights), `os-crowdsec`
* **Baseline Config:** Stateful firewall, default deny rule, no VLANs for this test, no IDS/IPS enabled for initial throughput baseline.
**Performance Benchmarks (iperf3, 60-second tests):**
| Test Scenario | Avg. Download | Avg. Upload | CPU Load (System) | Latency Add |
| :--- | :--- | :--- | :--- | :--- |
| **1 Stream, 1518 Byte MTU** | 940 Mbps | 941 Mbps | 12% | <0.2ms |
| **10 Streams, 1518 Byte MTU** | 942 Mbps | 943 Mbps | 18% | <0.3ms |
| **1 Stream, 512 Byte MTU** | 887 Mbps | 885 Mbps | 35% | <0.4ms |
| **10 Streams, 512 Byte MTU** | 898 Mbps | 901 Mbps | 68% | <0.6ms |
| **With Sensei DPI (10 streams)** | 921 Mbps | 918 Mbps | 82% | <1.2ms |
**Key Observations:**
* **Throughput:** The system easily saturates a 1 Gbps pipe with standard MTU frames. The performance drop with smaller (512-byte) packets is expected due to increased per-packet processing overhead, but remains well within acceptable margins for a gigabit connection.
* **CPU as Bottleneck:** As packet size decreases, CPU load increases significantly. This is the critical data point for anyone planning to run deeper packet inspection (like Suricata in IPS mode) or numerous VPN tunnels. The J6412 handles basic routing at 1 Gbps with immense headroom, but that headroom is consumed by advanced services.
* **Data Quality Consideration:** For my lab's data pipelines, consistent low latency is crucial. The sub-millisecond added latency under pure routing/NAT conditions is negligible. However, enabling Suricata (not shown in table) added 3-8ms of jitter under full throughput, which is a trade-off requiring careful evaluation based on the sensitivity of streaming analytics workloads.
* **Power Efficiency:** Idle power draw was ~8W, peaking at ~14W during line-rate tests. From a total cost of ownership perspective, this platform is highly efficient for a 24/7 deployment.
**Configuration Snippet (Sensei DPI for application visibility):**
My use case requires identifying and prioritizing analytics traffic (e.g., from Kafka Connect workers). Sensei's application identification allows for precise traffic shaping.
```shell
# Example of a floating rule to tag traffic for a specific internal analytics host
pass in quick on igb1 inet proto tcp from {192.168.1.100} to any tag "ANALYTICS_OUT" keep state
```
A subsequent traffic shaper rule can then prioritize the `ANALYTICS_OUT` tag.
**Conclusion:** OPNsense on modest x86-64 hardware is more than capable of handling a 1 Gbps fiber connection with all basic services (firewall, NAT, basic QoS). The primary consideration for data professionals is the substantial CPU overhead introduced by smaller packets and advanced security services. For a reliable, low-jitter data pipeline, I recommend running IDS/IPS on a separate mirrored tap if possible, rather than in-line on the firewall itself, unless your hardware significantly exceeds the processing power of this tested unit.
--DC
data is the product
Interesting setup, and thanks for posting the hard numbers. That Protectli box with the J6412 looks like a solid pick for this.
I'm curious about the jump in CPU load from 12% to 18% when you went from 1 to 10 streams. It suggests there's still some per-connection overhead even with hardware offloading enabled. Have you tried any tests with Sensei's web filtering or IPS features turned on? I found the performance hit can be pretty steep once you start enabling those deeper packet inspections on a gigabit line, even on similar hardware. That's usually the real-world trade-off.
Always testing.