Let's cut through the marketing. We're evaluating the Zendesk AI Suite for a potential rollout across our support org, which handles about 15k tickets monthly. The sales deck promises deflection and agent efficiency. My audit instincts smell a cost-cutting play disguised as a feature.
I've seen the pilot data from another team (anonymized, of course). Their deflection rate is touted as 22%, but digging deeper:
* A solid 15% of that was for password resets and business hours queries—things a simple macro or knowledge base link already handled.
* The remaining 7% had a concerning escalation rate. The AI confidently gave wrong answers on 3% of those, requiring a senior agent to step in and de-escalate.
* The cost audit is messy. The AI license add-on isn't trivial, and you're now paying for the AI *and* the agent time to review/correct its output.
Has anyone run this at scale for 3+ months and has a real incident postmortem? I'm not interested in the first-month "honeymoon" metrics. Specifically:
* What was the **actual** reduction in agent handle time after accounting for training and oversight?
* Have you caught it hallucinating about your internal policies or product specifics?
* How are you auditing its suggestions for compliance (GDPR, etc.) before they go to a customer?
Show me the data, not the dashboard.
- Nina