I've been running a private Poe server for my team's internal tooling and documentation queries for about six months now. The problem we kept hitting was the classic "jack of all trades, master of none" with a single, general-purpose assistant bot. Someone would ask a detailed Terraform question and get a mediocre answer, or ask for a Prometheus alert rule and get a generic snippet that didn't follow our conventions.
So I built a dispatcher system. The core idea is a single 'router' bot that acts as the front door. It doesn't answer questions itself. Instead, it analyzes the user's prompt and routes it to a specialized, purpose-built bot. This has drastically improved answer quality and reduced follow-up clarification loops.
Here's the basic flow:
* User messages the `team-infra-router` bot.
* The router bot runs a classification prompt to determine intent.
* Based on the intent, it silently calls the appropriate specialist bot via Poe's "bot as a service" feature.
* The specialist bot's response is delivered directly to the user.
The router bot's prompt is the critical piece. It's not doing the work, it's just making a routing decision. Here's a stripped-down version of the classification logic I use:
```
You are a routing classifier. Your ONLY task is to analyze the user's query and output a single keyword from the list below. Do not answer the question. Do not add commentary. Just output the keyword.
Output one of: terraform, kubernetes, monitoring, cicd, general
Classification guidelines:
- terraform: Questions about Terraform code, modules, providers, state, or cloud resource provisioning.
- kubernetes: Questions about Kubernetes manifests, Helm charts, ArgoCD, pod scheduling, or service configuration.
- monitoring: Questions about Prometheus, Grafana, alerting rules, metrics collection, or log aggregation (Loki/ELK).
- cicd: Questions about GitHub Actions, Jenkins pipelines, Docker builds, artifact management, or deployment stages.
- general: For everything else, including team documentation, process questions, or non-technical queries.
User Query: "{user_query}"
```
Then, in the router bot's settings, I have the "Bot as a service" calls configured. Each of those keywords maps to a different, specialized bot that I've also created.
For example, the `terraform-specialist` bot has a system prompt that starts with "You are an expert in Terraform for AWS and GCP. You always write code in Terraform v1.5+ syntax. You follow our internal module structure, which prefers..." and it has access to our actual Terraform module documentation via a document retriever.
The benefits have been concrete:
* **Accuracy:** The Kubernetes bot knows we use Kustomize over raw Helm for patches, and its answers reflect that.
* **Context:** The monitoring bot is pre-loaded with our actual Prometheus rule examples and alertmanager configuration snippets, so its suggestions are operational.
* **Maintenance:** Updating a specialist bot's knowledge (like adding a new CI tool) doesn't require retraining or confusing a monolithic model.
The main pitfall is the initial classification step. You need to tune the router's prompt with real examples from your team to catch edge cases. We had to add a "general" catch-all because the router was trying to send "When is the team offsite?" to the Kubernetes bot. Also, there's a slight latency overhead from the two-hop process, but for us, the quality gain far outweighs the extra half-second.
It does require maintaining multiple bots, but each one is simpler and more focused. If you're using Poe for more than casual chat and have distinct domains of questions, this pattern is worth the setup time.
Automate everything. Twice.
That's a really smart approach to a common problem. Your point about the router prompt being "the critical piece" is spot on. The classification step is everything. A misclassified query can be more frustrating than a generic one.
I'm curious about how you handle ambiguity. What happens when a user's prompt legitimately spans two domains, like asking for a Terraform config that also includes a specific Prometheus exporter? Does your router prompt instruct it to pick a primary specialist, or is there a mechanism to combine outputs?
Keep it constructive.
The classification step being "everything" assumes the router prompt itself is flawless, which is a huge assumption. You're just moving the single point of failure upstream.
How are you validating the router's classification accuracy? Are you tracking misrouted queries, or is this all based on a vague feeling of improvement? Without measuring precision/recall on the routing decisions, you're just swapping one black box for another.
Data skeptic, not a data cynic.
That's a fair critique. "Moving the single point of failure upstream" is a good way to put it. The router's quality is the linchpin.
You're right that validation is key. In our case, we did start with a simple log review, checking if the routed bot's first response was on-topic or if it had to ask for clarification. It wasn't perfect, but it gave us a baseline. Over time, we refined the prompt based on those logs. It's less about achieving a flawless router and more about whether the overall system produces better results than the single generalist bot did.
The goal isn't to eliminate a black box, but to have a simpler, more focused one that's easier to tune. The improvement feeling isn't vague; it's measured in fewer rounds of "I need more context" from the bots and higher satisfaction scores from the team. Still, your point about tracking misroutes is well taken. How would you suggest measuring precision in a setup like this without it becoming a huge manual audit burden?
Keep it constructive.