Many support teams are now touting their AI deflection rates, but I've found the raw numbers often obscure more than they reveal. A reported "40% deflection" can mean entirely different things depending on what's being counted, the time period, and the quality threshold applied. To have a meaningful conversation with vendors or to benchmark internally, you need a standardized method for extraction and analysis.
I recommend building a simple, repeatable report that isolates three core data points from your help desk platform. You'll need to pull this data monthly for a clear trend line.
* **Deflection Attempts:** The total number of instances where the AI feature (e.g., chatbot, suggested articles) was presented as a potential solution before ticket creation. This is your denominator.
* **Successful Deflections:** The subset of those attempts that resulted in a session closure *without* a ticket being logged. Crucially, you must filter out sessions that timed out or were abandoned; only count intentional exits.
* **Deflection Quality Sample:** A manual review of 50-100 successfully deflected sessions per month. Categorize the resolutions: Was the correct answer provided? Was the user likely to return with the same issue?
The extraction method varies by platform. For instance, with Zendesk, you would primarily use the Explore API, querying the `ticket_events` dataset for chatbot interactions and joining it with the `tickets` dataset to confirm non-creation. For ServiceNow, you'd likely start with the Interaction Context API. The key is to script these queries so the data pull is consistent.
Presenting this data alongside your overall ticket volume and cost-per-ticket metrics allows you to calculate true operational impact. A high deflection rate with low quality scores may simply be driving up repeat contacts and agent handle time on more complex tickets, negating the perceived savings. Without this disciplined breakdown, you're negotiating on faith, not figures.
— Jessica
Trust but verify. Then renegotiate.
That's a great start. How do you actually filter for the intentional exits? Most analytics dashboards just show "session ended" or "user left page". Is there a common event name or click you look for to count a successful deflection?