>any new package request needed a documented rollback and removal playbook *before* approval
That's brilliant. We do something similar, but we automated the check. Our CI pipeline won't pass a PR that adds a package unless the playbook includes a specific 'pkg delete' step. It's just a grep check, but it forces the thinking.
The dry-run on a test instance saved us too. We actually found it caught template logic errors that static validation missed, like when a variable substitution resulted in a duplicate rule the syntax checker didn't flag.
Data is the new oil - but it's usually crude.
Yep, and that recurring line item is the tax for escaping click-ops hell. The real question is whether management sees it as a tax or an investment.
We got the budget signed off by framing it as a product dependency, like any other library. The pfSense API version went into our internal product catalog with a support SLA. When a new version drops, we treat it like a dependency update: test in staging, update the 'package', regression test the automations.
If you don't formalize it, the hours just vanish into the void each quarter and everyone wonders why the automation is "brittle." It's not brittle, it's just unsupported.
That's a really interesting mindset shift. I'm still pretty new to this side of things - we mostly just use the GUI for our single office setup.
When you say you started pushing configs via code, did you find the API hard to get into? I've poked at it a little but it seemed a bit daunting compared to just clicking around. Was there a specific use case that made you take the plunge?
It's great to see someone else arriving at that same conclusion about it being an automation node. That shift in perspective changes everything.
The specific trigger for us was managing IPsec tunnels to dozens of cloud VPCs. Clicking through that many tunnels in the GUI for every change became a huge time sink and error-prone. Once we scripted the tunnel creation via the API, the time spent dropped dramatically and the configs became consistent.
Just be prepared for the initial learning curve. The API documentation is functional, but you'll likely spend some time with a tool like `curl` or Postman figuring out the exact JSON structure for things like firewall rule creation. That upfront investment pays off quickly at scale.
Review first, buy later.