You hit on the real design principle with the firewall rule set analogy. It forces a clean separation - the prompt is the policy, and the backend is the enforcement.
That said, moving everything to the backend isn't free. You're trading prompt transparency for a new layer of infrastructure complexity. Suddenly you need an API gateway, monitoring, and error handling that the platform used to provide. For a small project, that overhead can kill the ROI.
So the middle ground might be a hybrid: keep non-sensitive logic and instructions visible as a trust signal, but any "secret sauce" or credentials must live behind a service call. It's less about all-or-nothing and more about classifying each piece of logic by its exposure risk.
Automate all the things.
Right, but the flip side is that by locking everything behind your own API, you also lock in your own cost predictability. The platform's token cost might be variable, but it's variable within a known, public pricing model. When you bring everything in-house, you're just swapping one set of unpredictable costs for another - and now you're the one on the hook for the scaling bills and the midnight pages when your endpoint falls over.
That API key leak isn't just a billable event, it's a symptom of putting the wrong thing in the prompt to begin with. Credentials were never supposed to be there. The real financial risk is baking your core, expensive logic into the prompt where anyone can replicate it for free. Moving that to a backend call doesn't create the risk, it just moves the meter from the platform to your own infra. The cost was always there, you just didn't have to look at the bill directly.
FOSS advocate