Another week, another "agentic workflow" framework drops. The demos are always the same: a flurry of LLM calls, some tool-using, and a triumphant booking of a fake dinner reservation. Meanwhile, my inbox fills with alerts about cost overruns and latency spikes.
So, let's audit this. For the average line-of-business app—think invoice processing, inventory updates, customer support triage—what does an "agent" actually give you over a well-structured, deterministic service?
* **Unpredictable Cost & Latency:** Chaining multiple LLM calls isn't a workflow; it's a budget hazard. Each step is a non-deterministic, potentially expensive API call. A simple `if` statement is faster and cheaper than asking an LLM to decide the next step.
* **Operational Nightmare:** How do you monitor this? Tracing a single user request across a cascade of LLM decisions and tool calls is a distributed tracing problem on steroids. Good luck writing a clear postmortem when your "agent" went into a loop because of a poorly formatted API response.
* **The Compliance Shadow:** Data gets passed through multiple external services. Have you mapped all those data flows for GDPR/CCPA? Where's your audit trail for the decision logic? "The LLM chose to pull the customer's full history" isn't going to satisfy an auditor.
Most business logic problems are about data validation, state transitions, and integrating with existing APIs. You can solve 95% of them with a solid orchestration layer (yes, even serverless functions) and *maybe* one focused LLM call for a specific classification or extraction task.
```python
# Example: "Agentic" invoice processing vs. structured approach
# The "Agent" Approach (simplified):
# 1. LLM: "Analyze this PDF and decide what to do."
# 2. LLM: "Extract fields."
# 3. LLM: "Validate against database."
# 4. LLM: "If valid, post to ERP."
# The Boring, Auditable Approach:
def process_invoice(pdf_bytes):
# 1. Deterministic extraction (specialized parser/ML model)
invoice_data = extract_with_form_recognizer(pdf_bytes)
# 2. Rule-based validation & enrichment
validated_data = apply_business_rules(invoice_data)
# 3. **Single, guarded LLM call for exception handling only**
if validated_data["confidence"] < 0.9:
exception_note = llm_verify_discrepancy(validated_data)
log_for_human_review(exception_note)
else:
post_to_erp(validated_data) # Deterministic, logged, traceable
```
I'm not saying agents have no use. They're fascinating for open-ended research or creative tasks. But shoehorning them into transactional business logic feels like using a rocket launcher to change a lightbulb. The complexity, cost, and compliance overhead are rarely justified.
So what does this mean for your stack? Before you refactor, ask:
* What specific decision in my process *requires* non-deterministic reasoning?
* Can I bound the LLM usage to a single, well-monitored call?
* How will I explain this architecture's data flow and costs during an audit?
Or are we just building Rube Goldberg machines because it's the new trend?
- Nina
- Nina
Preach. That "operational nightmare" point hits home. I just spent two weeks trying to instrument an agent workflow in GitLab CI, and the traces were useless spaghetti. Trying to correlate a pipeline failure to a specific bad decision in a chain of five LLM calls? Forget it. A deterministic service logs its state; an agent just logs a series of opaque prompts and completions.
You're also dead on about cost. My team ran a test last month - a simple multi-step classification task. The deterministic service costs pennies per 10k runs. The "agentic" version, with its "reasoning" steps, was nearly 300x more expensive and 50x slower. For what? To occasionally misclassify anyway because the model got confused by an edge case.
I think the hype is pushing people to reach for these frameworks when a well-defined state machine with a single, focused LLM call at the fuzzy front-end would solve 90% of the problem without the chaos.
pipeline all the things
You've put your finger on a critical gap between the demo reel and production reality. Your mention of the "compliance shadow" is especially important - it's often treated as a post-launch checklist item rather than a core design constraint.
I've seen teams get lured in by the flexibility, only to realize later they've built a system where you can't definitively say what personal data was processed by which external model and when. That's a deal-breaker for many industries. The deterministic service you described logs a clear, auditable state change, which is what compliance officers and ops teams actually need to see.
—HR
Exactly. That compliance and audit trail issue is where deterministic systems don't just outperform agentic workflows, they make them non-starters for regulated data. I had a client in healthcare payments trying to use an agent to redact PHI from documents before analysis. The problem wasn't just the occasional failure; it was the impossibility of producing an audit log that satisfied their legal team. You can't log "the model decided not to redact this SSN" in a way that's actionable for an auditor.
The real architectural mistake is using LLM calls as a *substitute* for business logic, rather than a bounded, auditable *input* to it. A better pattern is to use a single, well-logged LLM call to extract structured data (like classification or entity extraction) and then feed that into a traditional workflow engine. That gives you a clear boundary: you can see the exact prompt sent, the raw completion received, and then the deterministic logic that ran afterward. The system's state is always known.
It turns a compliance nightmare into a manageable, scoped compliance concern for just one component.
connected
That "clear boundary" idea is really helpful. It sounds like you're saying to treat the LLM almost like an external API call in a microservice, where you can put all your logging, tracing, and cost control around that one call.
But in a beginner context, how do you actually enforce that pattern? Is it just a design principle, or are there frameworks or tools that help keep the LLM call isolated like that, so teams don't accidentally start chaining things?
That's a perfect way to frame it. The idea of a "clear boundary" is exactly right.
I've seen teams try to solve the audit trail problem by logging every LLM thought verbatim, but that just creates a mountain of unusable noise. What you need is a clear checkpoint: the state before the call, the exact input, the raw output, and then the state after your deterministic logic processes it. That checkpoint is something you can actually build compliance gates around.
Your healthcare example is a great one because it shows the stakes. An auditor doesn't want a transcript of the model's reasoning; they need to know the rule that was applied and see the evidence that it was applied consistently.
Keep it constructive.
Exactly. That checkpoint you described is the key to building something you can actually own and operate. I think the noise problem happens when teams log for debugging instead of logging for accountability.
It reminds me of a project where we used the single LLM call to produce a signed, immutable "proposed action" with a confidence score. Our deterministic logic would only accept proposals above a certain threshold, and the whole transaction, from raw input to final state change, was a single log entry with a clear accept/reject flag. An auditor could pull that log and immediately see the rule applied, the evidence, and the outcome.
It's less about the technology and more about defining that contractual boundary, like a service-level agreement for the AI's contribution to the process.
—HR
The point about alerts for cost overruns is what I live with daily. It's not just the spike, it's the unpredictable scaling. A deterministic service you can autoscale on queue depth or CPU. An agentic workflow's bottleneck is external API latency and token burn, which your cloud provider's metrics won't see until the bill arrives.
Your "compliance shadow" is real, but I'll add an ops angle: it makes incident response impossible. If a user reports a bug, I need to replay the exact logic path. With an agent, I'm replaying a stochastic process. I can't tell if it's a systemic flaw or just a bad roll of the dice that time. That's not a system, it's a liability.
shift left or go home
Totally agree. That "bounded, auditable input" pattern is what actually ships to production. I've built it in GitHub Actions with a step that's just a container running a script that calls the LLM API, logs the exact prompt and raw response as a step output artifact, and then passes structured JSON to the next deterministic job.
The trick is making that step the *only* allowed path for LLM calls in the entire pipeline. Enforce it with a linter or a policy-as-code check. Otherwise, someone will inevitably sneak in a "quick" direct call later and blow up the audit trail.
It turns the LLM into just another data source, like a database query, which is how it should be treated.
pipeline all the things