Hey everyone! 👋 I'm deep into evaluating Synthesia for a potential integration into our content workflow, and I'm about to start a trial. I know from past experience with other APIs that it's easy to get dazzled by the demo and miss the practical, day-to-day metrics that really matter for reliability.
I want to set up a proper testing framework from day one. Beyond the obvious "video quality," what specific technical and operational metrics should I be logging?
My initial list includes:
* **API Response Times:** For both generation status checks and the final asset delivery webhook.
* **Webhook Reliability:** Percentage of successful payload deliveries vs. missed calls. I'll be testing with a dedicated endpoint that logs everything.
* **Error Rate & Types:** Categorizing `4xx` vs. `5xx` responses, and specific codes for quota issues or malformed requests.
* **Render Time Consistency:** Does a 1-minute video take roughly the same time at 2 PM as it does at 2 AM?
* **Cost per Unit:** Tracking credit/video length ratios against the stated pricing model.
I'm also curious about the **connector quality** for platforms like Zapier or Make. Has anyone stress-tested them? For example, I'd set up a scenario like this in Make to test error handling:
```json
{
"scenario": "Webhook Retry on Failure",
"steps": [
"1. Synthesia webhook ā Make hook",
"2. Parse JSON for video URL",
"3. Router: If status = 'completed', proceed to Google Drive",
"4. Router: Else, wait 5 min, then re-poll API"
]
}
```
What else am I missing? Especially around rate limiting headers, data formatting for custom avatars/voices, or unexpected workflow bottlenecks you've hit.
ā chloe
Webhooks or bust.