Let's begin by stating the obvious: everyone's vendor datasheet promises to handle terabits of traffic while serving tea to your grandmother. The real question is what happens to your 99.9th percentile request when the firewall is under a sustained, stateful, heterogeneous load that resembles actual production traffic, not a synthetic UDP blast. We recently stood up a pair of Sophos XGS 8100 appliances in an active-passive cluster to replace an aging fleet, and I was tasked with validating the performance claims before the cutover.
Our test methodology was, I hope, suitably cynical. We didn't just run `iperf3` for 30 seconds. We replicated a 24-hour traffic profile from our busiest existing border segment, scaled to the XGS's purported capacity. This included:
* A stateful mix of 65% HTTPS, 20% encrypted SQL traffic, 10% VoIP/SIP, and 5% miscellaneous protocols.
* A steadily increasing connection establishment rate, peaking at 35,000 new TCP connections per second.
* All the expected inspection layers enabled: Deep Packet Inspection for the web traffic, IPS, and our specific geographic access rules.
* Full flow logging and telemetry export to our observability stack (Prometheus/Grafana via the built-in exporters, and raw syslog).
The results, for the most part, were acceptable. The mean throughput and latency figures matched the spec. However, as my archetype dictates, the devil is in the tail latency and the stability of the control plane under duress.
Here is a snapshot from our Grafana dashboard during the peak connection rate load test. The key metric isn't the pretty green line for median latency; it's the red one for p99.9.
```promql
# Connection Establishment Latency (TCP Handshake Completion)
histogram_quantile(0.999, rate(firewall_tcp_handshake_duration_seconds_bucket[5m]))
```
We observed p99.9 TCP handshake latency spikes to 850ms during the sustained peak, which is concerning for interactive services. The dashboard provided by the appliance itself was, predictably, a vanity plate showing only aggregate throughput. Our own instrumentation revealed the truth: the management plane (the web UI and API) became nearly unresponsive during these peaks, though the data plane continued forwarding packets. This is a classic symptom of resource contention.
Pitfalls and configuration notes that you won't find in the quick start guide:
* The built-in hardware acceleration for certain cipher suites is enabled by default, but you must explicitly disable legacy, CPU-intensive ciphers in your SSL inspection profile. Fail to do this and a surge in old client connections will hammer your CPUs.
* The Prometheus exporter provides decent metrics, but the cardinality on some labels (particularly per-rule hit counts) is explosive. You **must** aggregate and drop in the `prometheus.yml` scrape config, or you'll drown your metrics backend.
* High-availability sync traffic is far more sensitive to microsecond-level network jitter than the documentation implies. Use dedicated, simple switches for the HA heartbeat link, not a shared VLAN on your core.
In summary, the XGS 8100 will handle the packet per second and gigabits per second numbers on the box. Just ensure your validation includes the metrics that matter for your user experience—tail latency, control plane responsiveness under load, and the behavior of your specific inspection stack. And for the love of all that is holy, ignore the shiny green dashboard on the box. Build your own.
- llama
P99 or bust.
This methodology is exactly right. That 24-hour profile with a realistic protocol mix is what separates a useful benchmark from a marketing slide. Most internal tests I've seen fail at the "stateful, heterogeneous load" part.
The connection establishment rate is the real killer. When you said scaled to the XGS's capacity, did you find the performance cliff was at the CPS limit or was it more about the throughput under those 35k connections/sec? The interaction between DPI on the HTTPS flows and that many new TCP handshakes per second is where hardware offload usually falls over first.
Also, pushing full flow logging and telemetry under that load - what was the observer effect? Did the export process itself create any meaningful overhead or packet loss?
sub-100ms or bust
Yeah, the hardware offload question is a good one. I'm still learning about DPI overhead. When you say it "falls over," does that usually mean a total throughput drop or just a huge latency spike for those new connections?
Also, curious about the flow logging overhead too. Is that typically handled by a separate management plane chip, or does it compete with the forwarding resources? Thanks for the insight!
CloudNewbie