Everyone wants to talk about AI agent capabilities, but I wanted to see how they handle the inevitable faceplant. So, during a recent security review for a chat-based agent platform, I added a simple line to the RFP: "Provide your standard incident response plan for agent malfunctions, including hallucinations, data leakage, or inappropriate actions."
You'd think I asked for the nuclear codes. The responses fell into a predictable pattern of enterprise sales evasion.
First, two of the five (the smaller "agile" vendors) didn't even acknowledge the request. Their proposals were 80% feature lists and 20% pricing, zero mention of operational risk.
The three that did respond gave me variations on the same theme:
* One copy-pasted their generic SOC 2 incident response doc. It mentioned "servers" and "databases" but the word "agent" or "hallucination" appeared nowhere.
* Another provided a high-level flowchart that ended with "notify customer and escalate to engineering." No SLAs, no definition of severity levels specific to agent behavior, no clarity on rollback procedures.
* The third gave me a marketing paragraph about their "robust AI safety framework" and "continuous monitoring," but when pressed for a concrete runbook, the sales engineer said it was "proprietary" and I'd get access "upon signing."
This isn't just paperwork. If your agent spits out confidential data or makes a wildly inappropriate recommendation, you need to know exactly how the vendor will shut it down, how they'll notify you, and what their root-cause analysis entails. The fact that none of them had a ready, actionable plan tells me this is an afterthought. They're selling you the rocket, but haven't built the abort system.
I'm now adding specific, non-negotiable requirements to my evaluation rubric: a dedicated agent incident playbook with defined severity triggers (P1: data leak, P2: harmful hallucination, etc.), explicit communication SLAs, and a provision for post-mortem reports. If they can't provide it during the procurement cycle, they're eliminated.
The gap between sales demos and operational reality is a canyon.
Your mileage will vary