I'm evaluating Relevance AI's platform for potential internal automation, specifically for IT helpdesk workflows like password resets. The promise of an AI agent that can execute multi-step workflows via API calls is intriguing from a cost perspective, as it could reduce the load on our tier-1 support staff.
However, I'm struggling to map their marketing examples to a concrete, secure implementation for this use case. Has anyone successfully built and deployed such a flow? My primary concerns are:
* **Cost Structure:** Their pricing is based on "AI Units." How many units would a typical password reset consume? This involves parsing a user's email, verifying identity against an HR system (via API), generating a temporary password, and updating the directory (e.g., Azure AD via Microsoft Graph). A single workflow could easily involve 4-5 LLM calls plus the orchestration logic. At scale, this could become a significant variable cost compared to a fixed-cost traditional ticketing system add-on.
* **Security & Guardrails:** How do you strictly confine the agent's actions? For instance, preventing it from resetting passwords for users in a "privileged accounts" OU in AD. Is this managed entirely within the prompt, or are there structural permissions within Relevance AI's workflow builder?
* **Error Handling & Cost Control:** If an API call to the HR system fails, does the agent retry? Who bears the cost of the failed LLM calls used up to that point? Setting budget alerts seems crucial.
A basic conceptual workflow might look like this in their builder, but the cost and reliability details are opaque:
```yaml
Trigger: User emails 'helpdesk@company.com' with "reset password"
Steps:
1. Extract user identifier from email body (LLM).
2. Query HR API to verify employment status and manager (Tool Call).
3. Send verification code to manager via Teams (Tool Call).
4. Upon confirmation, generate temporary password & update AD (Tool Call).
5. Send new password via secure channel (Tool Call).
```
I'm interested in comparisons with building a similar agent using Azure OpenAI with Logic Apps or AWS Bedrock with Step Functions, purely from a total-cost-of-ownership and operational burden standpoint. Is the abstraction Relevance AI provides worth the premium in AI Unit costs for a high-volume, repetitive workflow like this?
Less spend, more headroom.
> Cost Structure: Their pricing is based on "AI Units."
This was the blocker for us, too. We built a proof of concept for a similar workflow, and the cost per run was too unpredictable for a high-volume task like password resets. Those "AI Units" add up fast when you have chained LLM calls for decision points.
On your security question, guardrails are defined in the workflow logic itself, not as a separate agent policy. You'd have to explicitly code a check to query the AD group membership before the reset step. If that check fails, you abort the workflow. It works, but it puts the security burden entirely on your implementation being airtight.
terraform and chill
The cost is the main reason we never moved past POC. For password resets, you need predictable, near-zero marginal cost per execution. LLM-based "AI Units" for a high-frequency task is a non-starter.
You've also hit the core security issue. The guardrail is just another step in the same AI workflow. If that LLM call glitches or gets tricked, your entire control is bypassed. That's not a guardrail, it's a suggestion.
Beep boop. Show me the data.