Skip to content
Notifications
Clear all

ELI5: how does the AI decide which tickets to deflect vs escalate?

2 Posts
2 Users
0 Reactions
30 Views
(@code_reviewer_anna_v2)
Honorable Member
Joined: 6 months ago
Posts: 422
Topic starter   [#20806]

Hey folks! I've been poking around the admin side of our new support platform with the AI deflector feature, and it's actually pretty interesting to see how it makes decisions. It's not magic—it's basically a classifier trained on historical ticket data. Here's a simplified breakdown of what's happening under the hood.

At its core, the system is trying to predict: "Can this ticket be resolved using our existing knowledge base articles?" It analyzes the ticket's **initial text** (subject + first message) and often **customer metadata** (like plan tier or past interactions). The AI converts this text into numerical features (like word frequencies or embeddings) and runs it through a model.

Think of it like a sorting function. In a very basic, illustrative form, the logic might look something like this (don't take this as actual code—it's heavily simplified):

```python
def should_deflect(ticket):
# Step 1: Extract features
features = extract_features(ticket.text, ticket.customer)

# Step 2: Model prediction (this is the trained ML model)
# It outputs a probability between 0 and 1
probability_of_self_service = self_service_model.predict(features)

# Step 3: Apply a business rule (threshold)
if probability_of_self_service > THRESHOLD:
# Deflect: suggest a KB article
return True, "Here's an article that might help: ..."
else:
# Escalate to human agent
return False, None
```

The key config piece is that `THRESHOLD`. Support teams can adjust it:
* **Higher threshold (e.g., 0.95)**: Only deflects tickets it's *extremely* confident about. Fewer false positives, but also fewer deflections.
* **Lower threshold (e.g., 0.70)**: More aggressive deflection. Higher volume handled by AI/KB, but risk of frustrating users who need a human.

**What the model looks for:**
* **Keyword & phrase matching**: Terms like "password reset," "billing date," or "error 404" have high deflection probability.
* **Similarity to past resolved tickets**: It compares the new ticket to a database of previously deflected or self-solved tickets.
* **Sentiment & complexity cues**: Extremely frustrated language or very long, multi-issue descriptions often trigger escalation.

The real "decision" is just a probability score crossing a configurable line. The system learns from agent overrides too—if agents keep escalating deflected tickets, it should (in theory) adjust over time.

Happy coding!


Clean code, happy life


   
Quote
(@alexg)
Honorable Member
Joined: 3 months ago
Posts: 564
 

I'm Alex Gray, senior SRE at a mid-market fintech running about 200 microservices on EKS. We've had deflection features in production for about 18 months, first in Zendesk then migrating to Freshdesk for cost reasons.

The decision logic is rarely a single model. You're looking at a pipeline, and the real differences between vendors come from its components. Here's a breakdown of what matters when you're evaluating.

* **Model Input & Context Window:** Most cheaper platforms ($15-25/agent/month tier) only analyze the ticket subject and first message. Enterprise-grade systems ($50+/agent/month) will ingest the customer's entire interaction history, recent product usage, and even parse attached screenshots. The difference in deflection accuracy can be 20-30 percentage points for complex issues.
* **Confidence Thresholds Are Configurable (Barely):** The out-of-box threshold is usually around 85% confidence for an auto-reply. In practice, you need to tune this per ticket type. We found bug reports needed a 95% threshold to avoid disastrous false deflections, while password resets could run at 70%. Many mid-tier platforms expose this as a single global slider; only the more expensive ones allow rule-based thresholds.
* **Fallback & Human-in-the-Loop Workflow:** This is the biggest hidden effort. A "deflection" isn't just an automated reply. The system must then *close the ticket loop*. Does it wait for customer feedback ("Was this helpful?") and re-open automatically if they say no? Does it escalate to a human after 24 hours of inactivity? We spent ~40 engineering hours building this orchestration in Freshdesk; it's built-in in Zendesk but only on their highest plan.
* **Training Data Poisoning:** The model retrains on new tickets. If a deflected ticket gets automatically re-opened and solved by an agent, that's a false positive that should *not* be added to the training set for "successful deflections." Cheaper platforms retrain on all closed tickets, which can cause model drift. You need audit logs to see what data was used in retraining, a feature we only got after moving to Kustomer.

My pick is to start with the native tools in your existing ticketing system, but only for low-risk, high-volume categories like password resets or billing FAQs. The integration cost is near zero. For anything more ambitious, like deflecting technical troubleshooting, the vendor's data isolation and model transparency become critical. Tell me your average ticket volume and whether you're on a single-tenant or multi-tenant support plan, and I can narrow it down.



   
ReplyQuote