Just stood up a new OPNsense VM for a branch office on Azure. Spec'd it with 2 vCPUs and 4GB RAM, thinking that's plenty for basic routing and the IDS/IPS (Suricata) I planned to enable.
Big mistake. The moment I turned on Suricata for a few internal interfaces, performance tanked. Packet inspection isn't free.
Here's what I saw in the dashboard and logs:
* System load average spiked to 15+.
* The VM's RAM usage pinned at 100%, heavy swap usage.
* Suricata logs showed `packet dropped` warnings due to `lack of resources`.
* Actual throughput dropped to a crawl, under 100 Mbps on a 1 Gbps link.
The issue? Suricata needs RAM for its ruleset and packet capture buffers. My rule set was about 50MB, but the live traffic buffer allocation was insufficient. The system was constantly swapping, killing performance.
I resized the VM to 8GB RAM. Immediate difference. Suricata stable, load average back under 2, throughput where it should be.
**Lesson:** Don't treat your firewall VM like a lightweight container. If you're running IDS/IPS, especially on multiple interfaces or at high throughput, RAM is your primary constraint, not CPU. My sizing was based on cost-saving assumptions from a purely routing perspective, which was wrong.
For anyone virtualizing, start with these minimums if you plan to use IDS/IPS:
* **Base (routing/NAT only):** 2 GB
* **With IDS/IPS enabled:** 8 GB
* **With high-throughput (>1 Gbps) or many rules:** 16 GB+
The Azure bill for that B4ms (4GB) vs. B8ms (8GB) jump is real, but the alternative is a non-functional security layer. Budget for the RAM.
show me the bill
show me the bill
Oof, that's a hard lesson to learn, especially when you're trying to be budget conscious. Thanks for sharing the specifics on the load averages and the packet drop warnings, that really paints the picture.
When you resized to 8GB, did you also need to adjust any of Suricata's own memory-related settings, or was the VM RAM bump alone enough to resolve the `lack of resources` warnings? Just curious how much is the VM and how much is tuning the service itself.
Glad you got it sorted! This is super helpful for my own planning.