I'm loving the Relevance AI platform for automating our internal reports, but my `generate_text` tool keeps spitting out different formats each time—sometimes markdown, sometimes plain text, sometimes with weird headers. Makes my downstream parsing fail 😅
Here's a snippet of my current, problematic setup:
```yaml
- tool: generate_text
instructions: "Summarize the monthly cost changes. Include percentage and absolute change."
output_variable: cost_summary
```
I've tried being super explicit in the instructions, but the structure still wanders. Has anyone nailed down a bulletproof prompt or config pattern for consistent, machine-readable output?
Maybe a system prompt tweak? Or a different tool? Would love to see your working examples!
#savings
Yeah, you're hitting the classic issue with free-form text generation. It's a language model, not a templating engine. Your instruction is too vague on format.
For machine-readable output, you need to lock down the structure in the prompt. Specify the exact format you want, line by line. I treat it like a code contract. Here's what works for me:
```yaml
- tool: generate_text
instructions: "Summarize the monthly cost changes. Output MUST be in this exact format. Do not include any headers, markdown, or extra text.
Format:
Service: [service_name]
Previous: [amount]
Current: [amount]
Absolute Change: [amount]
Percent Change: [percentage]%
Separate each service with a blank line."
output_variable: cost_summary
```
If that's still flaky, I'd skip `generate_text` entirely for structured data and use a combination of `extract` and `calculate` tools. Pull the raw numbers into variables, then format them yourself in a later step. It's more pieces, but it's deterministic.
Automate everything. Twice.
Oh, that's a super helpful example. Treating the prompt like a strict contract makes a lot of sense. I've been struggling with similar inconsistency.
What happens if your summary needs to include a note or an exception that doesn't fit the rigid format you defined? Does the model sometimes still add that commentary and break the parsing? Or does your "MUST be in this exact format" instruction kill any flexibility?
Just my two cents.
The core issue is you're instructing for content but not form. user31's contract approach is correct, but it's often insufficient alone. You need to combine it with two other controls: a system-level instruction and post-processing.
First, add a system prompt to your agent that sets the baseline: "You are a data formatting engine. Your outputs must be plain text with no markdown, headers, or commentary unless explicitly requested."
Second, your generate_text instruction must define a formal grammar. For cost changes, I'd specify a delimiter-separated format that's trivial to parse, like CSV. This is more reliable than expecting consistent line breaks.
```yaml
- tool: generate_text
instructions: "Summarize the monthly cost changes. Output must be valid CSV with exactly these columns: Service, Previous, Current, Absolute_Change, Percent_Change. Use a dollar amount for Previous and Current. Use a dollar amount for Absolute_Change. Percent_Change must be a number followed by '%'. Do not include a header row. Do not add any other text."
output_variable: cost_summary
```
Even with this, you should still validate and clean the output with a simple code step - treat the LLM as a probabilistic parser. The real fix is architectural: don't rely on a single, free-form generation step for structured data.