I'm currently evaluating text-to-speech providers for an interactive voice response (IVR) system where latency is a critical performance metric. The use case involves generating short, dynamic responses (under 10 seconds of audio) in real-time, where end-to-end latency directly impacts user experience.
My procurement team has shortlisted WellSaid Labs and Play.ht based on voice quality and their advertised streaming capabilities. However, I'm finding a lack of independent, apples-to-apples benchmarking data on their actual performance in a real-time context. Vendor-provided figures are often optimal lab results.
I'm seeking data points or shared experiences on:
* **Cold Start Latency:** Time-to-first-byte for a first-time request to a new voice.
* **Warm Start Latency:** TTFB for subsequent requests using a cached voice or model.
* **Consistency:** Variance in latency during peak vs. off-peak hours.
* **Geographic impact:** Differences observed when generating audio from various global regions relative to their service endpoints.
If anyone has conducted formal load testing or has operational metrics from a production environment, I'd be particularly interested in the methodology used (e.g., measuring from a specific AWS region, audio length, sample rate). Any insights on how their respective streaming APIs handle network instability would also be valuable for our total cost of ownership model, which factors in potential fallback strategies.
— Jessica
Trust but verify. Then renegotiate.
Totally feel your pain on the vendor-provided figures being lab results. I ran some informal tests last year for a conversational AI prototype, though it wasn't a full-scale load test.
From my notes, WellSaid's cold start for a new voice was consistently in the 1.8-2.5 second range for a short sentence, which was a bit higher than expected. Their warm start, once you've used a voice in the same session, dropped reliably to about 800ms. Play.ht had a wider variance on cold starts - sometimes under a second, sometimes pushing three seconds - but their warm performance was more comparable.
The real kicker for something like an IVR might be geographic impact. My requests from Europe to WellSaid's primary endpoint added a pretty steady 300-400ms round-trip, just on the network hop. Play.ht's distributed endpoints helped there if you can route regionally. Have you considered baking in a short pre-fetch or a connection pool to mask some of that cold start time?
api first
Geographic variance is the real data point everyone skips. Those numbers mean nothing without knowing your data center region and their edge locations.
> baking in a short pre-fetch or a connection pool
You're just adding complexity to fix their architecture. If you need a pool for an IVR, you've already lost. The problem is using a remote service for real-time.
Their cold start times are basically "we spin down your container." No amount of your code will fix that.
Keep it simple