I've been testing these features for a few months now at my company. The data is pretty clear: our deflection rate for internal IT tickets (password resets, software installs, VPN issues) is over 70%. For customer-facing support? It's below 20%, and satisfaction scores drop when the AI jumps in first.
My theory is internal staff have shared context and simpler, procedural problems. The AI can pull from internal wikis and follow exact steps. But a customer's question is often vague or emotional. The AI misreads it and gives a generic, frustrating reply.
Has anyone else seen this split in performance? I'm curious if we should just turn it off for customer tickets.
Hey there, user853. Hugo here. I'm a systems and integrations lead for a 250-person SaaS company, and we run both types of auto-reply: an internal AI chatbot for IT that pulls from our Notion knowledge base, and an external one on Intercom for general customer support, so I'm seeing this exact split daily.
Here's a breakdown from our setup:
1. **Deflection Rate and Query Type**: We see an 80% resolution rate for internal IT tickets (password resets, Slack channel adds). For customer support, deflection is under 25%. The key detail is 90% of successful internal queries are for documented, step-by-step procedures. Vague customer questions like "why is my export slow" or emotionally charged tickets immediately break the flow.
2. **Knowledge Base Integration and Maintenance**: Internal wikis are controlled, structured, and updated in batches, maybe weekly. Our AI uses a single Notion workspace with clear ownership. The external customer-facing knowledge base is a sprawl of help articles, past ticket snippets, and forum posts. Keeping the AI's context accurate requires near-daily tuning and costs us about 5 engineering hours a week in vetting.
3. **Cost of Failure Profile**: A misstep internally costs maybe 10 minutes of an employee's time before they click "get human help." A bad AI reply to a paying customer escalates the ticket to a "complaint" tier 60% of the time, adding 15+ minutes of agent work to de-escalate, which directly hits our support ops budget.
4. **Configuration and Training Overhead**: The internal bot needed about 40 hours of initial setup to map our IT processes and link permissions. The customer-facing bot took triple that to build guardrails, sentiment triggers, and escalation paths, and it still needs monthly retuning on a sample of failed conversations.
My pick is to keep it on for internal IT and move to a "human-first, AI-assist" model for customers. We switched our external tool to only suggest agent replies and draft answers after the ticket is triaged, which kept satisfaction scores stable. To give a cleaner recommendation, tell us your current CSAT baseline and what percentage of your customer tickets are actually repetitive, like "where's my invoice?" versus complex troubleshooting.
hugo