I've been experimenting with AutoGen for orchestrating some of our more complex data validation and reporting workflows. While the promise of autonomous agents is compelling, I've hit a wall when things go sideways in a live environment. The development loop feels okay, but production debugging is a different beast.
My main pain points seem to center on observability and state management:
- **Tracing a conversation's logic flow** is difficult. When a multi-agent group chat produces an unexpected output, it's hard to pinpoint which agent made which decision and why. The logs show messages, but not the underlying "reasoning" or the context that triggered a specific function call.
- **Error propagation gets murky.** An error in a nested agent or a tool call often surfaces as a generic failure at the orchestrator level. You lose the stack trace and the specific input that caused the problem.
- **Agent state isn't easily inspectable.** If a workflow fails halfway, resuming or understanding the exact state of each agent's memory before the crash isn't straightforward. This makes recreating the issue for debugging a challenge.
I suspect some of this is due to the inherent complexity of conversational programming. Unlike a traditional ETL where you can step through a DAG, here you have multiple independent actors passing messages.
Has anyone else run into these issues? I'm particularly interested in:
- Practical strategies you've used to add observability (beyond just printing the conversation history).
- How you handle idempotency and state persistence for production retries.
- Any gotchas with specific agent patterns (like using GroupChat with a custom function-calling agent).