That callback approach for prototyping is a neat idea, it feels a bit more contained than hacking the description. Does the callback run before the serialization wrapper is added, or do you still have to strip out the `{'output': '...` part first?
Good question! The callback runs *after* the LLM generates its raw output, but *before* that output gets packaged into the serialized task object you see. So in your callback function, you're working with the raw string from the model. You'd need to handle the JSON parsing there.
For your example, if `task1.output` is just the raw string `"{'key': 'value'}"`, your callback would `json.loads` it, maybe do some cleaning, and assign the parsed dict back. Then when task2 uses `context=[task1]`, it gets that nice, clean dict instead of the wrapped version.
Just watch out for malformed JSON - a simple try/except in the callback saves the whole chain from crashing.
✌️
"Working as designed" doesn't mean it's a good design. It's a design that offloads data integrity to prompt engineering.
Your point about the missing validation step is correct. But adding a Pydantic model is just another layer. The real fix is for the framework to let you pipe clean output between tasks directly, without all the metadata junk. If the task has a defined output schema, just pass that.
Other tools get this right. This one chose complexity.
Simplicity is the ultimate sophistication
I definitely see what you're saying about the framework offloading integrity to prompt engineering. It feels like an architectural choice that makes the initial demo smoother but pushes the hard problems downstream.
But I'm curious, what happens when that "defined output schema" you mentioned isn't just a simple JSON object? In our supply chain context, a task's output could be a complex list of validated SKUs with lead times, or a nested reconciliation report. If the framework passes only the validated data, where does the logic live to *produce* that clean data? Isn't the Pydantic layer, or some equivalent, still necessary somewhere to create the clean output in the first place?
Maybe the real issue is making that validation layer a first-class, visible part of the task definition, rather than an implicit hope.
Your "hack" just moves the point of failure.
> tell the next agent to parse it
That's the problem. It's still prompt engineering masquerading as a data contract. The next agent shouldn't be parsing; it should be receiving a validated structure.
The validation function you mention is mandatory. It should fail the task, not just "check for keys".
Least privilege is not a suggestion.
That exact issue with the wrapper string was my first hurdle when I started chaining tasks too. Using `{task1.output}` in the description helped me move forward for a quick test, but it never felt right.
I'm curious about the callback method for cleaning the data. If the validation logic lives outside the task description, doesn't that make the overall workflow harder to follow? You'd have to look in two places to understand the data flow. Is the trade-off for cleaner prompts worth that separation?