I've been digging into Zendesk's AI features lately, specifically trying to get a clear picture of how effective our new automated answers and suggested macros are at deflecting simple tickets. The promise is less repetitive work for agents, but the measurement part is proving tricky. My main headache? Making sure a single ticket interaction isn't counted multiple times across different deflection metrics, which inflates our success rate and gives a false sense of efficiency.
From what I can tell, there are at least three places where deflection-like metrics appear, and they can easily overlap:
* **Answer Bot** deflection, based on clicks on its suggested articles.
* **Flow Builder** automation, for tickets resolved by a bot workflow.
* **Macro suggestions** to agents, where an agent uses an AI-suggested macro to solve a ticket quickly (sometimes considered a "soft" deflection).
If a customer clicks an Answer Bot article *and* that interaction triggers a Flow Builder automation that resolves the ticket, is that counted twice in a generic "deflection" dashboard? I want to measure true, unique ticket avoidance.
Here’s my current approach, and I'd love the community's feedback on its validity or if there's a better path. I'm focusing on creating a single, reliable report.
1. **Define "Deflected Ticket" Clearly:** For our team, I'm defining it as any ticket that is solved without ever entering an agent's *working* queue (i.e., no agent actively picks it up, modifies it, and replies). This includes tickets closed via web widget automation, Flow Builder, or community forum solves.
2. **Key Data Source - the Ticket Solve Event:** I'm relying heavily on the `ticket.solved` event in Explore. The critical step is checking the `ticket.solved via` attribute. I then filter for values like "web widget," "trigger," or "automation."
3. **Deduplication is Key:** I create a calculated metric that counts **unique ticket IDs** where the solve event meets my "deflected" criteria. This prevents any single ticket from being counted more than once, no matter how many AI interactions happened on its journey.
4. **The Denominator Question:** To get a rate, I'm still debating the best denominator. Is it "all solved tickets" in the period? Or "all tickets created"? I'm leaning toward "all tickets created," as it shows the overall volume reduction attempt.
Has anyone else built a consolidated Explore report for this? My biggest worry is whether I'm missing a scenario where an agent action and an automation blur together, or if there's a timestamp issue I haven't accounted for. Concrete examples from your own dashboards would be incredibly helpful.
grace
The right tool saves a thousand meetings.