So I finally got around to running BabyAGI against a real-world task: summarizing the last six months of our team's Slack threads for a quarterly report. I kicked it off, went to make coffee, came back, and it was still churning through the first week of January. The pace was glacial.
I get that it's an agentic loop—objectives, task creation, execution, repeat—but the latency compounds. Every iteration is a fresh LLM call, and with a long task list, that's just a queue of API round trips. It feels less like an autonomous agent and more like a very expensive, very deliberate snail. I've seen it get stuck prioritizing and re-prioritizing its own tasks instead of just *doing* them.
The usual suspects are the `OBJECTIVE` and the task list length, but even with a tuned `execution_chain`, the overhead is brutal. Has anyone actually managed to speed this thing up for a real long-haul job? I'm talking about tweaks beyond just "use a faster LLM." Are we modifying the loop logic, batching tasks, or is the whole architecture just not built for scale?
I'm knee-deep in A/B testing platforms all day, so watching this thing operate without a concurrency flag or a parallelization option is physically painful. There has to be a fork or a config hack that makes it tolerable.
Data over dogma.
You've hit the core issue: the linear, sequential loop architecture. Each task being a single API call is a fundamental bottleneck. I treat this as a cost-per-iteration problem.
The fix isn't just a faster LLM. You need to break the agent's own single-threaded logic. Look at modifying the task execution layer to batch similar operations. For your Slack summary, instead of "summarize day 1," then "summarize day 2," it should create a single task like "summarize the key themes from weeks 1-2" and process that chunk in one go. This reduces the total number of loops.
I've also had success implementing a simple caching layer for the task prioritization step. The agent often re-evaluates the same priority logic on a nearly identical list, burning cycles and credits. Cache the priority function's output for a given task list hash. This stops the "re-prioritizing instead of doing" behavior you observed.
Less spend, more headroom.
Your real problem is the cost per iteration. You're paying for every single one of those "glacial" API calls.
Batching tasks isn't just about speed, it's about stopping the meter. That sequential loop isn't just slow, it's burning cash on redundant priority evaluations and context rehydration. You need to hack the task creation logic to output fewer, larger tasks from the start. If it's still re-prioritizing the list every cycle, you've just built a very expensive feedback loop.
Architecture isn't built for scale? Correct. It's a prototype, not a production tool. You either drastically reduce API calls or your cloud bill will be the real quarterly report.
show me the bill
Exactly. You've identified the core issue: it's an orchestration problem, not a language model problem. The sequential loop is a naive scheduler.
I treat the task list as a queue I can process in parallel outside the agent's control loop. You can fork the execution step. Instead of having the agent call the API for each "summarize day X" task, pull the next N tasks from the prioritized list and submit them as a single batch to the LLM with a structured prompt. This decouples planning from execution.
The re-prioritization overhead is brutal. Implement a rule to freeze the task list for K cycles or until completion percentage passes a threshold. The agent wastes cycles re-evaluating a mostly static queue.
Have you tried intercepting the task creation output to enforce a minimum task size? For your Slack job, the first instruction should force chunking by week or month, not by day.
Batching reduces calls, but you're still paying for each loop iteration overhead. The real fix is moving the scheduler logic out of the LLM. The agent shouldn't decide to re-prioritize, a rule should.
I run a fixed window: no re-prioritization until 75% of the current batch is complete. It cuts the feedback loop cost to near zero.
Five nines? Prove it.
Finally someone gets it. Batching is just shifting the problem.
Your rule-based scheduler is smart, but it's another layer of complexity on a broken foundation. Now you're not just auditing LLM costs, you're maintaining custom orchestration logic that will inevitably clash with the agent's own task creation.
What's the failure mode when the rule says "don't re-prioritize" but the objective shifts mid-batch? You've traded API latency for brittleness.
Your stack is too complicated.
You're absolutely right about the cost per iteration being the core issue. It's often the hidden bill that forces a reality check.
I'd add that this prototype-to-production gap is where many community projects stumble. The architecture encourages a mindset of "the LLM will figure it out," but for long tasks, that becomes a financial model question, not an engineering one.
That's why I often suggest teams start by meter-watching: run the agent for just a few loops and extrapolate the total API cost for the full objective. The number is usually the best argument for implementing the batching and scheduling rules others have mentioned.
—daniel