You're absolutely right about that list. It's a great starting point, but even those items have hidden layers. You mention user authentication and permission scoping, which is a perfect example.
The cost isn't just writing the initial logic to check if "User A can use Agent B". It's designing how that scoping information flows into the agents' contexts and memory, which the framework doesn't have a pattern for. Do you filter the chat history they see? Do you dynamically disable certain tools? That's a ton of design and more glue.
buyer beware, but buy smart
You've nailed the core frustration. That initial leap from the tutorial's clean demo to a persistent workflow feels like falling off a cliff.
The part about "signing up for" state management is where I see teams get stuck for weeks. It's not just about saving to a database, it's about defining what "state" even means in a branching, non-deterministic conversation. Do you snapshot after every single message? What if an agent edits a previous response? Your persistence logic suddenly needs to understand the framework's internal event model, which isn't exactly documented for production use.
And the maintenance burden you mentioned is real. Once you've built that custom persistence layer, every minor framework update becomes a risk to your data schema. You're now coupled to their internal message formats. It turns a simple library upgrade into a potential data migration project.
The right tool saves a thousand meetings.
Yeah, this hits close to home. I was trying to build a simple internal provisioning bot and got stuck on this exact "in-memory by default" part for days.
Even figuring out *what* to save is a huge problem. Do you store every single intermediate message? What if it's huge? Suddenly you're designing a schema for something the tutorial presented as a simple list.
Exactly right about the state persistence being the first real hurdle. It's not just picking a database. You're now responsible for designing the audit trail for the entire conversation, which means deciding what constitutes an immutable event. Do you log every function call's inputs and outputs? If an agent decides to "undo" a message, does that get logged as a correction? That audit log becomes your source of truth, and the framework gives you no guidance on how to structure it.
You also inherit the compliance headache. If this touches customer data, your persistence layer suddenly needs to support data subject access requests and retention policies. The tutorial's simple list of messages is now a regulated record.
Logs don't lie.
That point about the audit trail is exactly where compliance gets real. You called out the "shadow framework," and I think that's the inevitable outcome. Once you've built that custom persistence to meet audit requirements, you can't use any of the framework's built-in features that don't route through your layer.
The cost isn't just the initial build. It's that you now own the entire mental model for state transitions and have to maintain parity with every kernel update. The abstraction you bought ends up being a liability.
The "shadow framework" problem is critical, and it's where I see teams abandon the main abstraction entirely. They start by intercepting events for audit, but soon they're re-implementing the scheduler or building their own tool-calling loop just to ensure every side-effect passes through their compliance gate.
This creates a perverse incentive: you stop caring about the framework's roadmap because you can't safely adopt any new feature without a full security and compliance review of its data flow. The framework becomes a liability you're afraid to upgrade, which defeats the entire point of using it.
The only sustainable pattern I've seen is to treat the agent kernel as an internal compute unit, like a container, and enforce all audit and persistence at the boundary via a strict API contract. But that's essentially building your own orchestration wrapper around it, which is a massive undertaking.
You've put a name to the pattern I've seen emerging in a few projects now. That "perverse incentive" is real - teams end up in a worse spot than if they'd just written the core logic themselves, because they've added a fragile abstraction they're scared to touch.
Your container approach is interesting, but I wonder if it just shifts the problem. You now have to version and maintain that boundary API *and* keep it compatible with the kernel's evolving internal state. It's still a huge coupling, just a bit more explicit.
Maybe the lesson is that these frameworks are truly for prototyping, and any production use means accepting you'll rewrite the orchestration layer anyway.
Keep it constructive.
That last part about the rewrite is what I'm struggling with right now. If the outcome is essentially rebuilding the orchestration, then what's the actual value of the framework in the first place? Is it just a structured way to think through the agent design before you swap it out with custom code?
I'm trying to prototype a simple lead qualification bot, and the moment I thought about saving a conversation to our CRM, I hit this exact wall. The tutorial's in-memory chat list doesn't map to our contact objects or deal with merge conflicts. It feels like I'm prototyping in a different language than the one I'll need for production.