Alright, so I finally pulled the plug after letting SuperAGI chew on a single RTX 4090 for a solid quarter. Wanted to see if the hype about "open-source AGI" held up for actual, you know, *work*, not just demos.
My setup was pretty standard:
- **Hardware:** Single RTX 4090, 64GB RAM, i9-13900K.
- **Use Case:** Primarily automating research workflows—web scraping with analysis, summarizing technical papers, and generating code for data pipelines.
- **Deployment:** Local via Docker Compose, using their main `stable` branch.
The good? The agent orchestration is legit. Setting up a multi-step workflow with tools is intuitive. The built-in tools (Google Search, Filesystem, Code Interpreter) are surprisingly robust once you get them configured. Performance for a single complex agent was decent—no major memory leaks that crashed the system, which is more than I can say for some other frameworks I've tested.
The not-so-good? The "single GPU" part is where the cracks show. Don't expect to run a swarm of heavy agents concurrently. The moment I tried to scale beyond three concurrent agents with medium-complexity tasks (e.g., one scraping, one writing, one coding), the VRAM got saturated and everything slowed to a crawl. Also, the default models (GPT-based via API) are fine, but swapping in a local Llama 3 70B via Ollama? Good luck without serious tuning. The framework assumes a certain latency and response format that not all local models handle well.
Here's a snippet of the config that *actually* worked for a stable single-agent setup:
```yaml
# super_agi_config.yaml
LLM: openai # Switched to local_llm (ollama) for some tests
MODEL: gpt-4-turbo
TEMPERATURE: 0.7
MAX_TOKENS: 4000
PERFORMANCE_MODE: balanced # 'speed' just caused timeouts
TOOL_TIMEOUT: 120
```
Biggest pitfalls:
* **Resource Monitoring is Essential:** You **need** `nvtop` or similar running. The framework isn't great at queuing tasks when resources are low—it'll just try and fail.
* **Tool Latency:** The Google Search tool can block an agent for *minutes* if the query is broad. Had to implement custom timeouts.
* **Persistence:** The SQLite DB got corrupted twice after abrupt shutdowns. Migrated to Postgres, problem solved.
Verdict after 3 months: It's a powerful playground and viable for *sequential* complex tasks on a single GPU. But if you're thinking "AutoGPT on steroids for production," you'll need a GPU cluster or deep pockets for API bills. The promise is there, but the "AGI" part is more of a modular, capable assistant than any true autonomous intelligence.
benchmarks or bust
Totally agree on the single GPU bottleneck. It's not just VRAM - the queuing system for concurrent agents needs work. I've had better luck running lightweight agents (just logic, no local LLM) and offloading heavy inference to separate cloud endpoints. Have you tried their Kubernetes config for scaling? It's rough but lets you pin specific agents to pods.
YAML all the things.
The cloud endpoint trick is a smart workaround for the queuing issue. We did something similar for a campaign analysis workflow - kept the planning/local logic agent on the local GPU and pointed its "heavy lifting" tasks (like summarizing large datasets) to a dedicated cloud LLM API. It cut our average task time by almost half.
But that Kubernetes config... yeah, it's rough. We got it running but the agent-to-pod pinning felt more like a band-aid than proper scaling. Did you find a clean way to handle the shared storage for agent outputs between pods? That was our biggest headache.
You're right about the queuing being the real issue, not just VRAM. The system tries to be too smart about scheduling and ends up creating a bottleneck even when resources are free.
Your point about lightweight local agents is key. That's the pattern that's worked for my team as well. We treat SuperAGI more as a central planning and routing brain, while any model inference, especially for large batches, gets sent to dedicated endpoints. It makes cost tracking per project much cleaner, too.
I haven't tried their Kubernetes setup because that hybrid local/cloud approach removed the immediate need. But I'm curious if you ran into issues with agent state persistence when you offloaded inference? We had a few cases where the local agent would proceed before the cloud call fully returned, corrupting the workflow.
—Anita