Everyone's scrambling to build "AI agents." Most are just over-engineered RAG pipelines with extra steps. Ran both AutoGen and LangChain for half a year on a real internal workflow. Spoiler: you probably need neither.
LangChain feels like a leaky abstraction factory. It's great for prototyping, but you end up wrestling its "chains" and "agents" in production. The lock-in is real. AutoGen's agent-to-agent chat model is interesting, but the orchestration overhead is massive. You're suddenly a manager for a team of bickering LLM instances. For 90% of use cases, you're better off with a simple script calling the API directly.
Our final "agent" for a doc-generation pipeline ended up being 150 lines of Python, no framework. It's more maintainable and debuggable.
```python
# Simplified core loop. No framework magic.
def process_query(query, context):
# 1. Decide action (simple rule or tiny model)
# 2. Call tool or LLM directly
# 3. Format response
# That's it.
```
Frameworks add complexity you might not need. Ask what you're actually automating before picking a tool.
Keep it simple
This is really helpful, thanks. I'm just starting to look at LLM automation for sales emails, and the idea of a simple script is appealing.
For your 150-line doc pipeline, did you run into issues managing state or retry logic? That's the part where I'd be tempted to reach for a framework.
The "bickering LLM instances" part made me laugh. I can see how managing multiple agents would get chaotic fast.
So you're saying that for a simple doc pipeline, the framework overhead wasn't worth it. That really hits home for me, since I'm always worried about overcomplicating things on a tight budget.
But when you say a simple script is better, how do you handle cost tracking for all those direct API calls? One reason I looked at LangChain was the built-in callbacks for token usage logging. Did you have to build that monitoring from scratch, and was it a pain?
Cost tracking is a fair point. But you don't need LangChain's whole abstraction layer just for that. The API response from OpenAI includes token usage in the JSON. A few lines to log it to a file or a metrics service is all you really need.
If your "tight budget" is the main concern, ask yourself what's more expensive: a few hours building a simple logging wrapper, or months of developer time wrestling with a framework's quirks? The vendor lock-in with these platforms can be a budget killer later, when you're forced into their upgrade path for features you could have built yourself.
That built-in callback system is nice in theory. In practice, you're paying for it with complexity and inertia. It's a classic enterprise trap, honestly.
—DW
Exactly. Vendor lock-in hits harder than people think. You'll see it when the next version of LangChain changes its callback interface and your production pipeline breaks.
If you're worried about cost tracking, just log to CloudWatch or Prometheus directly. The token counts are already in the API response, so you're just moving numbers. That's maybe 10 lines of code.
The bigger hidden cost is developer time adapting to their abstractions.
Show me the bill
That "150 lines of Python" number is the most telling part. You've hit on the core of the issue: frameworks create work to solve problems you might not even have yet.
My team tried the same approach on a compliance check pipeline. The initial LangChain prototype was quick, but adding a simple conditional branch to the logic meant untangling three different classes. Rewriting it as a straightforward function with a couple API calls took an afternoon and was immediately clear to the junior devs. The "leaky abstraction factory" is a perfect description - you spend more time sealing the leaks than moving water.
But I'll push back slightly on one thing: sometimes you do need the structure when scale becomes a real factor, not a hypothetical. If you're coordinating more than three distinct tasks with complex state, the "bickering LLM instances" problem emerges even in a custom script. The real trick is knowing when you've crossed that line, and most projects never do.
— skeptical but fair
I totally see what you mean about that line where scale flips the script. What's your threshold for "more than three distinct tasks with complex state"? Is it more about the number of steps, or how much they need to remember from previous interactions?
Our team is starting to look at a workflow that might involve checking data, drafting a summary, and then validating it - that's three things, but the validation step needs context from the first two. It feels like it's teetering on the edge of "should I structure this more?"
rookie
That's a great way to frame it. The threshold isn't really about the step count - it's about the coupling between steps. Your example of validation needing context from the first two steps is exactly the trigger.
We hit a similar point with a procurement workflow. The moment we needed to maintain and pass a nuanced, validated "summary object" between more than two steps, our simple function started sprouting global variables and awkward parameters. That's when we introduced a lightweight, plain-old-Python class to act as a state container. It's not a framework, it's just a structured way to pass context without things getting messy.
If your validation logic is simple - like checking for keyword presence - you can probably keep it functional. If the validator needs to reference specific data points or reasoning from earlier steps, you're already in state management territory. Start with a dedicated state object before considering a framework.
Agree on the state container. That's the first real fork in the road.
But a Python class can become its own kind of framework if you're not careful. I've seen teams over-engineer the "lightweight state object" into a bespoke monolith with its own DSL. Then you're right back to debugging your own bad abstraction.
The trick is to keep it data-only. A dataclass or a typed dict. The moment you add helper methods, you've started building LangChain.
Don't panic, have a rollback plan.
For retry logic, the `tenacity` library is my go-to. It's a decorator, so you just wrap your API call function. For state, I started with a plain dictionary but quickly moved to a Pydantic model - it gives you validation and a clear schema without becoming a framework.
You're right to feel that temptation! The key is to ask if the framework is solving YOUR specific state problem, or just giving you a general one you have to customize anyway. My doc pipeline's state was basically "source text, chunks, and summaries" - a simple list of objects. A framework's state manager would've been overkill.
The chaos with multiple agents is real. I found the tipping point is when they start modifying shared state concurrently. If you can design your sales email steps to be linear (draft, then review, then send), you can avoid that whole mess.
That "150 lines of Python" number is the most telling part. You've hit on the core of the issue: frameworks create work to solve problems you might not even have yet.
My team tried the same approach on a compliance check pipeline. The initial LangChain prototype was quick, but adding a simple conditional branch to the logic meant untangling three different classes. Rewriting it as a straightforward function with a couple API calls took an afternoon and was immediately clear to the junior devs. The "leaky abstraction factory" is a perfect description - you spend more time sealing the leaks than moving water.
But I'll push back slightly on one thing: sometimes you do need the structure when scale becomes a real factor, not a hypothetical. If you're coordinating more than three distinct tasks with complex state, that simple script can turn into a ball of mud just as fast.
Data skeptic, not a data cynic.
Ah, the ball of mud argument. Heard that one a lot right before a team builds their own worse, undocumented framework.
The "untangling three different classes" problem you described with LangChain is real. But the alternative isn't a 150-line script forever. It's a 200-line script, then a 400-line script, then a 700-line "helper module" with five people's conflicting ideas of state management. Suddenly you're debugging race conditions in your homegrown agent dispatcher, which is just a LangChain clone with worse error messages.
If scale is a real factor, you'll need structure. The question is whether you adopt a known, flawed structure or invent your own. At least with the former, the leaks are documented on StackOverflow.
Absolutely right about the hidden cost. We lost a week because `get_openai_callback()` was suddenly deprecated. That's a week of dev time not adding features, just changing our monitoring because the framework decided to.
> just log to CloudWatch or Prometheus directly
Spot on. The API response gives you everything you need. Our wrapper function for logging is literally 12 lines, and it's ours. No more surprises.
The adaptation fatigue is real. Every new junior dev needs to learn LangChain's specific vocabulary before they can even reason about our business logic. It adds a layer of indirection that's often more confusing than helpful.
Clean code is not an option, it's a sanity measure.
>months of developer time wrestling with a framework's quirks
That's the key tradeoff, but it's not symmetrical. Those months happen up front with a custom solution. The framework's "quirks" often become time sinks later, during unplanned outages or upgrades when you're trying to trace a problem through layers of abstraction.
For cost logging, you're absolutely right. The vendor lock-in is less about the logging itself and more about the ecosystem. Once you adopt a framework's callback system for one thing, it's easier to adopt it for the next. Soon, you're not just logging to their metrics, you're structuring your entire flow around their patterns. Your 12-line wrapper becomes 50 lines of adapter code.
We saw this with LangChain's chain callbacks. What started as convenient token counting became a constraint when we needed to log to a different metric format for our finance team. We had to rip it out.
The adaptation fatigue you mentioned is so real. It's not just the junior devs, either. I've watched senior engineers lose a full day to LangChain's changing abstractions, all to fix something that would have been a two-line change in a direct API call.
Your point about the 12-line wrapper is key. That's the sweet spot - just enough structure to be consistent, but transparent enough that anyone can see what it's doing. The moment you're debugging through four layers of inherited classes to understand why a prompt changed, you've lost the plot.
The deprecation surprises are the worst kind of technical debt. It's not a choice you made, it's a choice made for you, and you get to pay the bill.
Review first, buy later.