I am currently evaluating text-to-speech services for a latency-sensitive application involving dynamic content generation, and the inference speed—specifically the time from API call to receipt of the first audio byte (time-to-first-byte, or TTFB) and the total time for full audio synthesis—is a critical performance metric. While both PlayHT and WellSaid Labs produce high-quality output, their architectural approaches (as inferred from documentation) suggest potential differences in pipeline efficiency that would materially impact user experience in interactive scenarios.
I have conducted preliminary, informal tests using their respective standard APIs, but I require more rigorous, comparable data. My initial methodology involved:
* **Environment:** A dedicated cloud instance (us-east-1) to minimize network variance.
* **Sample Text:** A standardized 250-character paragraph of neutral prose.
* **Measurement:** Scripts using `curl` with the `-w` flag for timing details, and custom Python clients using `time.perf_counter()` to capture client-side latency.
* **Parameters:** Default voice, 24kHz mono output format for both services.
My initial observations, which should not be considered definitive, indicated a variance. For example:
```bash
# Simplified measurement approach for TTFB
curl -X POST "https://api.play.ht/v2/tts"
-H "Authorization: Bearer $API_KEY"
-H "Content-Type: application/json"
-d '{"text":"$SAMPLE_TEXT", "voice":"$VOICE_ID"}'
-o output.mp3
-w "time_namelookup: %{time_namelookup}ntime_connect: %{time_connect}ntime_appconnect: %{time_appconnect}ntime_pretransfer: %{time_pretransfer}ntime_starttransfer: %{time_starttransfer}ntime_total: %{time_total}n"
```
The `time_starttransfer` value here approximates TTFB. However, this single-point measurement is insufficient.
I am seeking community input to validate or expand upon these findings. A comprehensive benchmark would need to control for and report on:
* **Regional API endpoints** and their effect on latency.
* **Voice-specific performance,** as some voices may utilize different underlying models.
* **Concurrent request handling** and any scaling latency penalties or efficiencies.
* **The impact of text length** on the linearity of synthesis time.
* **Streaming response performance** versus monolithic file delivery.
Specifically, I am interested in whether anyone has performed systematic A/B testing under controlled conditions, accounting for:
1. Cold vs. warm model inference latency.
2. Network hop consistency (using tools like `mtr`).
3. The performance tier of the subscribed plan (e.g., pay-as-you-go vs. enterprise).
Subjective feedback on "speed" is less useful without the accompanying methodology. If you have undertaken such measurements, sharing your approach, sample size, and results—even in raw form—would be invaluable. Furthermore, insights into the internal architecture of either service (e.g., pre-loading of voice models, caching strategies, or use of speculative execution) that you may have gleaned from support documentation or engineering blogs would help explain observed performance characteristics.
That's a really interesting approach. When you say "initial observations," were you seeing a consistent winner between them on TTFB, or was it too variable? I'm curious because I've only done much simpler timing checks for my own use. Your setup sounds way more controlled.