Skip to content
Notifications
Clear all

Unpopular opinion: Their support docs are terrible. Am I wrong?

2 Posts
2 Users
0 Reactions
1 Views
(@cloud_infra_vet)
Reputable Member
Joined: 2 months ago
Posts: 134
Topic starter   [#20059]

Having recently completed a migration of a media processing pipeline to Runway, I feel compelled to address a critical, yet often overlooked, component of their platform: the documentation and support knowledge base. While the core generative models are undeniably powerful, the operational scaffolding—specifically the guidance for integrating these models into robust, production-grade systems—is, in my professional assessment, severely lacking. This creates a significant hidden cost and risk factor for any serious cloud architect.

The primary failure mode is the documentation's assumption of a simplistic, one-off user interacting solely via the web interface. For those of us tasked with building automated workflows, the gaps become immediately apparent. For instance, consider a common task: invoking their APIs via a serverless function with proper error handling and idempotency. The API reference lists endpoints and parameters, but offers little to no guidance on:

* **Rate limiting and quota semantics:** The exact headers, backoff strategies, and how limits interplay across different model families. Is it requests-per-second or tokens-per-minute? The docs are silent, forcing a trial-and-error discovery process that is unacceptable for system design.
* **Comprehensive error codes:** Beyond a generic "500 error," what are the specific, actionable failure modes? Is there a structured error response body for, say, content policy violations versus temporary service unavailability? This is crucial for building resilient observability and alerting.
* **Infrastructure-as-Code examples:** A glaring omission. There is no Terraform or CloudFormation example for provisioning and managing necessary resources (like IAM roles, secrets for API keys) in a repeatable manner. This pushes teams towards fragile, manual console configurations.

To illustrate, here is a Terraform module I had to write from scratch because no analogous example exists. The Runway documentation would typically just say "use your API key."

```hcl
resource "aws_secretsmanager_secret" "runway_api_key" {
name = "prod/runway/gen-2/api-key"
description = "RunwayML API Key for Gen-2 model inference"
}

resource "aws_iam_policy" "lambda_invoke_runway" {
name = "lambda-runway-invoke-policy"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = "secretsmanager:GetSecretValue"
Resource = aws_secretsmanager_secret.runway_api_key.arn
},
]
})
}

# The Lambda function itself would then need a robust implementation
# for HTTP client with exponential backoff, parsing opaque errors, etc.
```

Furthermore, the search functionality within their support portal is ineffective, often returning unrelated articles from 2022 that reference deprecated endpoints or UI elements. This forces reliance on community Discord channels, which, while helpful, are no substitute for authoritative, versioned, and searchable documentation maintained by the engineering team.

In an enterprise context, this deficiency translates directly into increased development time, security risks from ad-hoc configurations, and operational fragility. For a platform positioning itself as a foundational AI tool, the infrastructure and developer experience must be treated with the same rigor as the model research. As it stands, the burden of operational knowledge is disproportionately placed on the consumer. I am curious if others undertaking complex integrations have encountered similar hurdles, or if my experience is an outlier.



   
Quote
(@hannahc)
Active Member
Joined: 3 days ago
Posts: 4
 

You've hit the nail on the head with the "hidden cost and risk factor." That's the real frustration. It's not just that you have to figure out the rate limits through trial and error, it's that every minute spent on that is a minute you're not building the actual pipeline. Suddenly, your project timeline balloons because you're reverse-engineering fundamentals from the platform itself.

I see a parallel in some sales tools I've worked with. They'll have a fantastic automation engine, but the documentation on how their internal queueing works for outbound emails is vague. You only discover the real throttling limits after your first campaign gets flagged. It forces you to build all these extra monitoring and pacing safeguards you shouldn't need.

I wonder if this is a common growth phase for API-first companies. They pour everything into the core tech, assuming power users will just "figure it out," while the support structure lags way behind. Makes you wish they'd at least label parts of their docs as "beta" or "experimental" for production use.


hannah


   
ReplyQuote