Anyone actually using Relevance AI in production for customer support

I've been conducting an extensive evaluation of Relevance AI over the past quarter, specifically for augmenting a multi-channel customer support operation handling approximately 50,000 tickets per month. The platform's promise of a "workflow AI" engine for automating complex support tasks is compelling, but I'm seeking concrete, production-grade data beyond marketing case studies. My initial analysis reveals a significant gap between conceptual demonstrations and the realities of latency, cost, and operational integration at scale.

Our primary use case involves a multi-step workflow: classifying inbound support queries (email, chat), routing them to a vector-based knowledge retrieval system (our own RAG pipeline built on Pinecone and GPT-4), and then synthesizing a draft response with specific product data and policy references. We initially attempted to orchestrate this with a combination of LangChain and custom Python services on Kubernetes, but the maintenance overhead for state management and error handling was non-trivial. Relevance AI's visual builder appeared to offer a solution.

Here is a simplified YAML export of a representative workflow we built and are load-testing:

```yaml
workflow:
name: support_tier1_triage
steps:
- extract_customer_intent:
uses: relevance/classifier
inputs:
query: "{{customer_message}}"
categories: ["billing", "technical", "account", "general"]
- retrieve_kb_articles:
uses: relevance/vector_search
config:
dataset_id: "prod_knowledge_base_v2"
embedding_model: "text-embedding-3-small"
top_k: 5
- check_past_interactions:
uses: custom_code
runtime: python
script: |
# Calls internal CRM API
return get_last_five_tickets({{customer_id}})
- generate_draft:
uses: relevance/llm_chain
model: "gpt-4-turbo"
prompt: |
Using the retrieved articles {{retrieve_kb_articles.results}}
and customer history {{check_past_interactions.data}},
draft a response addressing: {{customer_message}}.
Focus on intent: {{extract_customer_intent.top_category}}.
```

My preliminary benchmarks, however, raise questions:

* **Latency:** The median workflow execution time for a non-trivial chain (3+ steps with external calls) is between 4.2 to 7.8 seconds. This is acceptable for async email but problematic for live chat where we target sub-2-second initial acknowledgments. The overhead of their control plane, even with steps theoretically running in parallel, is measurable.
* **Cost Transparency:** While the platform abstracts LLM calls, our internal calculations suggest a 15-22% premium over directly calling the same provider APIs (OpenAI, Anthropic), once the Relevance AI platform fee is factored in. This might be justifiable for the reduction in devops burden, but the pricing model becomes opaque with high-volume usage.
* **Observability & Debugging:** The built-in logging is sufficient for high-level success/failure rates but lacks the granularity we require. For instance, tracing a specific error through a failed vector search step requires correlating logs from three different systems. We've had to implement a secondary OpenTelemetry pipeline to get proper traces, which negates some of the "simplification" benefit.

My core questions for the community are:

* Has anyone successfully deployed Relevance AI workflows in a high-volume (>10k workflows/day) production environment for support, and what were the key stability hurdles?
* How are you handling idempotency and state recovery for failed workflows, particularly when external APIs (like a CRM or payment system) are involved? Does their "retry" logic suffice, or did you need to build a wrapper layer?
* Are you using their native vector store and datasets, or primarily integrating external data sources (e.g., your own Weaviate or PostgreSQL with `pgvector`)? Performance comparisons here would be invaluable.
* What does your monitoring stack look like? Have you found the native tools adequate, or have you also had to export metrics to Prometheus/Grafana or Datadog for actionable alerts?

The architectural promise is significant, but I am cautious about vendor lock-in for such a critical path. I am particularly interested in comparative analyses with alternative orchestration layers like Prefect, Temporal, or even a more minimalist approach with tightly coupled FastAPI services behind a message queue.