Hey everyone, been deep in building some multi-step workflows in Relevance AI lately and hit a common snag. I'm orchestrating a lead enrichment process where one step finds a contact, and a later step needs to remember a specific ID from that first API call to update the record. It's a tiny piece of data, but crucial for linking the actions.
In platforms like Salesforce or HubSpot, you'd often use a record ID field, but within a single Relevance AI workflow run, I need to pass small state bits between my own custom nodes. I've seen the `workflow_state` in the docs, and I've also thought about using the built-in database for a more persistent storage. But for temporary, in-workflow data that doesn't need to live forever, what's the cleanest pattern?
I'm curious about the real-world trade-offs. For example, is it better to use `workflow_state` for short-lived data and the database for things you might need to audit later? How do you handle errors or retries without losing that little piece of state? Any gotchas with data types or size limits? Sharing your experiences would be super helpful for nailing down a reliable approach.
Hey, I'm also pretty new to marketing ops at a mid-size SaaS company. We run our nurture streams and lead scoring in Relevance AI, hitting about 5k workflow runs a month, so I've wrestled with this exact state problem.
My comparison after trying both:
1. **Storage Duration**: `workflow_state` is strictly for that single execution and vanishes after. The database table persists until you delete it, which is better for debugging.
2. **Setup Effort**: `workflow_state` is just there; you use it. The database needs a table created first, which took me maybe 10 minutes to get the schema right.
3. **Error Handling**: If a workflow fails and retries, data in `workflow_state` is reset. With the database, you can write the state after a successful step, so a retry can read the same ID and continue from where it left off.
4. **Real Limitation**: `workflow_state` has a size limit (I think it's 10KB in my environment). It's fine for an ID or two, but if your state grows, it'll break. The database can handle way more.
I'd recommend `workflow_state` for simple, linear workflows where you just pass an ID to the next step. Use the database if you need audit trails, have complex retry logic, or might need that data for a separate process later. If you told us how often your workflows fail/retry and if other workflows need to read the same data, the choice would be obvious.
You're right to focus on the temporary use case. For your lead enrichment example, `workflow_state` is the correct tool. It's a key/value store scoped to that single run. Think of it like a short-term memory for the workflow.
The gotcha is you need to explicitly set and get it in each node's code. If a later step fails and the workflow retries from the beginning, that state is lost. That's actually fine for true idempotent processes, but if your "find contact" API call generates a new ID on each run, you'll create duplicate records.
If you need to survive retries, you have to write to the database after the first successful step. That adds latency and cost, but for 5k runs a month it's negligible. The size limit for `workflow_state` is generous, but treat it like a cache, not a data warehouse.
List price is for suckers