I've been evaluating Relevance AI for the last six weeks as a potential orchestration layer to replace a brittle homegrown system we use for processing client data. The premise is solid, and I'll be the first to admit their UI/UX is among the best I've seen in the AI workflow space. Building agents and workflows visually is intuitive, and my junior devs picked it up in an afternoon. That's the good news.
The bad news is that we hit a wall almost immediately when we tried to integrate it into a real development cycle. The fundamental, crippling omission is the complete lack of **version control, branching, and proper environment promotion** for your AI workflows. This isn't a nice-to-have for any team doing serious B2B work; it's the bedrock of safe iteration and change management. Let me give you a concrete example from our pilot.
We have a workflow that:
1. Ingests a client's raw product data.
2. Calls a classification agent.
3. Passes the result to a validation agent that checks against their business rules.
4. Outputs a cleansed dataset.
We needed to tweak the prompt in the validation agent to handle a new edge case. In Relevance, you just... edit the live agent. There's no "create a feature branch," no "stage these changes for testing." You are editing what is effectively production. We thought we could be clever and duplicate the entire workflow:
```yaml
# This is the *idea* of what we need, not actual Relevance config.
# We needed a 'development' variant that promoted to 'staging' then 'production'.
environments:
production:
workflow_id: prod_123
agent_version: 2.1
staging:
workflow_id: stage_456 # Copy of prod, for pre-release testing
agent_version: 2.2-rc1
development:
workflow_id: dev_789 # Branch for active work
agent_version: 2.2-dev
```
But that's a nightmare to manage manually. Duplication is not version control. It creates configuration drift, and there's no audit trail of who changed what, when, or why. We had a scenario where a developer and a data scientist made concurrent, conflicting edits to the same agent. One's changes were silently overwritten. The "Activity Log" is just a high-level audit trail, not a diff-able history you can revert to.
For a team of one, maybe this is fine. For any organization where:
* You have more than one developer.
* You need to test changes against a staging dataset before going live.
* You have compliance or governance requirements (which is every B2B client I've ever worked with).
* You plan to maintain and iterate on these workflows for more than a quarter.
...this is a non-starter. The lack of version control forces you into a "hope and pray" deployment model. It makes rollbacks a manual, error-prone reconstruction process. The slick UI becomes nothing more than a very pretty, very dangerous facade.
I'm posting this because their marketing and demos completely gloss over this critical operational aspect. They're selling the sizzle of building AI agents but ignoring the fundamental engineering discipline required to run them reliably. Until they introduce proper versioning, environment isolation, and a promotion model (Git integration would be ideal), this platform is only suitable for prototypes and hobbyists. It cannot be the backbone of a mission-critical data migration or client workflow system.
My team has shelved the evaluation. The risk is too high. We're back to the drawing board.
—BW
Migrate once, test twice.
That's a really specific and painful example, thanks for sharing. You mentioned needing to tweak a prompt for an edge case and just editing the live agent. That alone would make me nervous for any client-facing process.
I'm currently looking at orchestration tools for sales ops, and version control is top of my list for similar reasons. How did your team handle rollbacks when something went wrong after an edit? Did you have to manually recreate a previous state, or was there any kind of audit log to at least see what changed?