Anybody actually using BabyAGI in production for customer support?

I ran a very similar experiment last year. That initial excitement about the task-driven loop hits a wall pretty fast in production. Let me tackle your points directly.

For hallucination, you can't just limit actions. You have to bake in verification. I wrapped every data-fetching step (e.g., SQL query) with a secondary "review" task that checks the retrieved data against the original user query for consistency. It adds overhead but prevents it from inventing customer details. More critically, never let it *execute* a database write or send a message directly. All outputs should be drafts queued for human review.

On stability, the vanilla task queue will indeed get stuck. It tends to over-decompose. I modified the execution step to reject new tasks if the objective was already met by previous ones. You'll also need to implement a hard iteration limit and a "stuck detector" that looks for repetitive task descriptions.

The integration glue was the bulk of the work, frankly. The BabyAGI core became a small part of a larger system that handled API polling, state persistence, and piping drafts back to our helpdesk. If you're going down this path, consider building your agent logic as a module within a more robust orchestration framework like LangChain's or even a simple Celery queue, rather than trying to extend the original script directly. The brittle sentiment prototype you mentioned is the canary in the coal mine.

You're right about the integration work being the real beast, but calling the BabyAGI core a "small part" is generous. I found its contribution so marginal I ripped it out after two sprints. The task-driven loop is a fancy demo feature that gets in the way more than it helps for a deterministic workflow like triage.

Your verification step for data fetching sounds like a clever workaround, but doesn't it just move the hallucination risk one layer back? Now you're trusting the review task's judgment, which uses the same flawed reasoning engine. That's just adding more links to a chain that's already suspect.

The whole premise of forcing a customer support flow into an "autonomous" agent loop feels like fitting a square peg into a round hole because the round hole is trendy. Why not just build a well-structured pipeline with clear decision points? The answer is always "but then it's not an agent," which, frankly, is the goal.

That super basic prototype feeling brittle is the real warning sign. Scaling that to production would mean building guardrails, not just more tasks.

To your point on integration, yes - expect a lot of glue code. The demo loop doesn't know about your Zendesk webhooks or database schema. I budgeted for twice the dev time I thought I'd need for the connectors alone.

On stability, the queue does get weird. It might spin off tasks to "analyze sentiment" and "determine sentiment" back-to-back. You spend more time debugging its plan than reviewing its outputs.

The dream of it pulling customer data and drafting responses is exactly where marketing gloss meets real world risk. That loop is a liability magnet, not a feature.

You can't limit hallucinations, you can only accept them and build a human review cage around every single output. Every action, even a simple data fetch, needs a separate verification step using a different prompt and context. That doubles your token cost and latency before a human even sees a draft.

And for stability, "weird loops" is the default. The queue doesn't get stuck, it gets creative. It'll spawn ten redundant tasks to analyze the same "urgent" ticket. You'll spend more time babysitting the agent's existential crisis than you would just answering the tickets.

Building the custom glue for Zendesk or Intercom is the easy part. The hard part is accepting you're building a complex, expensive system to generate a first draft that you still have to check 100%. Is that really a win for a small team?

You're right about the token cost and latency doubling, but it's even worse. That verification step you mentioned? It's using the same model. The guardrail is made of the same brittle material as the track, just painted a different color. If the first action hallucinates a customer's subscription tier, the "verification" task often just hallucinates a convincing reason why that false tier is correct.

The real cost isn't just the extra API calls. It's the cognitive load on your team to now debug a two-stage stochastic process before they can even trust the data enough to write a reply. You've traded a simple, predictable workflow for a complex, unpredictable one that still requires the same human effort at the end. That's not engineering, that's just adding layers of expensive uncertainty.

Interesting point about the verification step. But if it's using the same model, wouldn't a truly bad hallucination pass both steps? Like if it invents a fake ticket number for the SQL query, the review might just confirm "yes, that number matches the query" even though the number itself is wrong.

You're hitting on the classic prototyping gap, where the core concept feels brittle in a simple test. That's a major red flag for production, especially with customer data.

> My code looked something like this for the objective

If you're already fighting brittleness just to classify sentiment, you're seeing firsthand how the task loop will struggle with your more complex goals of triage, data retrieval, and drafting. That brittleness compounds with every new action and verification step people are describing.

The real question might be simpler. Are you looking for a triage automation tool, or are you committed to the "agent" paradigm specifically? There are more deterministic, rule-based approaches to auto-tagging and data fetching that might get you 80% of the dream with 10% of the headache.