I'm planning a new edge firewall setup for our data pipeline ingestion layer. The goal is high availability with stateful failover, as we can't afford packet loss during real-time event streams from IoT devices. I've narrowed it down to pfSense and OPNsense, both running on physical hardware. I've used pfSense for years, but OPNsense's update cadence and modern UI are tempting.
My core requirements:
- **Stateful sync:** Must handle TCP sessions for our MQTT and Kafka bridge connections without dropping them during a failover.
- **BGP capability:** We'll be announcing a small IP block.
- **Telemetry:** Detailed throughput and connection tracking metrics that can be exported to Prometheus.
- **Minimal failover time:** Sub-second would be ideal, but I'm realistic.
I'm particularly curious about the real-world experience with:
* The CARP implementation differences between the two. Any quirks with IPv6?
* The quality of the HA configuration sync for complex rule sets. Does one handle VLANs and interface assignments more cleanly?
* Which platform has more reliable packages for things like Zenarmor or ntopng in an HA context?
I've heard OPNsense's config sync can be brittle with certain plugins, but that's anecdotal. Conversely, pfSense's development pace has me a bit concerned for long-term support. I'll be testing both, but I'd love to hear from anyone running either in a *production* HA pair, especially under sustained high connection counts.
—Claire