That point about the Datadog bill doubling hits home. We found a related issue: the volume of logs wasn't just a cost problem, it made the logs themselves useless for any kind of post-hoc analysis. When you're logging every intermediate step, you lose the signal in the noise, and querying for a specific transaction's path becomes impossible.
Your strategy to log only tool calls and final outputs is the right one. We enforce this at the SDK level by having a standard wrapper that strips out internal reasoning before emitting to our observability layer. It forces a clean schema.
However, you have to be careful with that approach in healthcare. If you're ever subject to an audit for a model's decision, logging *only* the final output might not satisfy the requirement to demonstrate the reasoning chain. You need a separate, secure, and potentially offline audit log for the full trace, but it shouldn't be your primary operational logging stream.
p-value < 0.05 or bust
You've zeroed in on the core tension: your expertise in structured CRM processes versus the exploratory nature of agent frameworks. Coming from a background with Salesforce, you're already thinking in terms of deterministic, auditable workflows, which is the correct lens.
The critical mistake I've seen repeatedly is teams using CrewAI or LangChain as the *orchestrator*. For your specified sequence of validate->transform->log->notify, that's a fundamental category error. These frameworks are optimized for dynamic pathfinding, not for guaranteeing step B only runs after step A succeeds with a full audit trail. You'll end up building all the monitoring and state management a tool like Prefect or Dagster gives you for free, but on a shaky foundation.
A concrete example from a recent engagement: we used a Prefect DAG to manage patient record synchronization. The "transform" step included a single, isolated function call to a finely-tuned LLM for normalizing disparate clinic note formats into a structured field. The LLM call was wrapped in a custom class that handled PII redaction before sending to the API and logged the input/output pair with a correlation ID. The agent framework's scope was limited to that one box in the flowchart, not the lines connecting all the boxes. Start by sketching your pipeline as a flowchart; if more than one box says "decide what to do next," you've probably over-applied the agent pattern.