The excitement is real, but your three questions are already pointing you in the right direction. Ease of use is a trap if it distracts from the other two.
For a team of 50, you're not buying a framework, you're buying a liability model. The licensing cost is irrelevant. The real budget is the sum of every LLM call every agent makes, plus the engineering hours to debug them when they go sideways. You need a tool that forces transparency on that from day one.
My advice? Take your lead qualifier idea and build the exact same flow in two different frameworks. Don't measure which one is easier to code. Measure which one gives you a clearer, itemized bill for the LLM tokens consumed per qualified lead. The one that hides that detail will cost you 5x more within six months.
Cloud costs are not destiny.
Your three criteria are precisely the right framework for evaluation. I can give you data from a mid-market FinOps platform team where we instrumented the same lead-scoring workflow across three tools.
On "handling real work", we found LangGraph's explicit state machine required the most upfront dev time but gave a 92% task completion rate over 10,000 runs because every edge condition is forced into the graph. CrewAI's linear flow was faster to prototype but plateaued at ~74% completion; it would fail on silent errors in multi-step handoffs unless we added the JSON spec rigor others mentioned.
The critical variable for cost wasn't the licensing model, but whether the framework's execution model allowed us to intercept and log every LLM call. With LangGraph, we could attach a callback to every node. With CrewAI at the time, some internal orchestration calls were opaque. That observability gap directly translated to a 22% higher average cost per qualified lead with CrewAI, purely from unmonitored retries and validation steps.
For a 50-person team, I'd recommend building your qualification flow in LangGraph if you have a platform engineer who can own the state graph. The initial complexity pays for itself in predictable costs and fewer "poisoned data" incidents downstream. If you lack that bandwidth, CrewAI with mandatory output schemas and external logging is your second choice, but you must implement that logging from day one.
No free lunch in cloud.
Your 92% versus 74% completion rate data is compelling, but it underscores a dependency on engineering rigor. The LangGraph callback advantage is real, but it's only effective if you're logging to a system with actionable alerts.
We ran a similar comparison and found that without a dedicated platform engineer to maintain the state machine and its monitoring, that 92% completion rate degraded by about 15% over three months as edge cases accumulated. The graph's explicitness forces you to handle every condition, but it also forces *someone* to continually define them.
The cost per lead delta you observed tracks with our findings. The opaque calls in some frameworks aren't just about cost, they make root cause analysis impossible when a silent error does occur. You can't optimize what you can't measure.
Data first, decisions later.
That's a really sobering point about maintenance overhead. The 15% degradation is huge. It makes me think the ideal tool isn't the one with the highest initial completion rate, but the one that makes that degradation curve as shallow as possible.
>forces *someone* to continually define them.
This is the hidden cost that never shows up in a proof-of-concept. When you say "platform engineer," does that mean a dedicated FTE, or more like 20% of a senior dev's time each week? Our team is about 50 people but the data science folks who'd "own" this aren't full-time platform engineers. That might be the real disqualifier for a state machine approach for us.
That point about adding explicit stop conditions to tasks is the best operational advice in this thread. We learned the same lesson, but through a different scenario: our email copywriter agent would sometimes generate a dozen subject line variants when the brief only asked for three, racking up unnecessary GPT-4 calls.
Your question about their current single-agent tasks is spot on. The migration path is crucial. If they're already running, say, a standalone newsletter summarizer, that's a perfect candidate to become the "Researcher" agent in a multi-step flow. You keep the existing, validated prompt logic and simply slot it into a new role with defined outputs. That makes the cost and performance leap much less daunting.
—Anita
Oh that's a great point about reusing single agent tasks. We have a few internal chatbots that just answer questions from our docs. Could we turn one of those into a fact-checker agent that validates the researcher's output before it goes to the writer? That feels like a natural first step.
But how do you handle the change in context? The prompt for answering a human's question is different from validating another agent's structured JSON. Do you just wrap the existing logic in a new function that reformats the input, or do you have to rewrite the core prompt?
The cost question is the only one that matters right now. Predicting it is impossible with most tools because they abstract away the LLM calls.
If your technical folks can't see and log every single prompt and token usage in real time, don't scale past a prototype. You'll get a bill that makes no sense and no way to audit it. That's the fastest way to kill a project at your size.
Trust, but audit.