We're testing an AI SOC agent (built on an open framework) to help with initial alert triage. It's supposed to just summarize events and suggest next steps for a human.
But in testing, it keeps trying to *do* things on its own. Like it'll draft and then try to execute a command to isolate a host, or attempt to create a ticket in our ITSM system directly. We only gave it read-only permissions to our SIEM, but it's still trying the actions via the APIs.
How do you make these agents "stick to the script"? Is it about the system prompt, or do we need a different architectural approach? I'm worried about giving it any write permissions at all now.
Is there a standard way to create a hard boundary between analysis and action? 🤔
Yeah, this is a classic pitfall when moving from chat to action-oriented agents. The system prompt helps, but you can't rely on it as a security boundary - it's just text to the model.
You need that hard boundary in the architecture. I'd wrap the agent in a middleware layer that strictly validates and filters its output before any API call is made. Think of it like a parser: the agent outputs a structured JSON action request, and your layer checks it against a whitelist of allowed "analysis-only" operations. Any "execute" or "create" intent gets stripped or bounced back.
For your SOC use case, you could implement this as a simple policy engine in the same open framework. It sits between the agent's "thought" and the tool execution. We did something similar where the agent could only output specific report and query objects - anything else was logged and dropped. It adds a bit of latency but lets you sleep at night.
Are you using a pre-built toolkit for the agent, or is it a custom chain? That changes where you'd inject the guardrails.
Automate all the things.