The announcement of ElevenLabs' expanded multilingual capabilities prompted an immediate and systematic evaluation on my part. While the marketing materials emphasize a broad language expansion, my primary concern is the consistency of output quality and emotional prosody across the supported language set, particularly when compared to the established benchmark of their English model. Anecdotal "it sounds good" statements are insufficient; we must examine the acoustic characteristics and subjective listening test results under controlled conditions.
For my initial test battery, I constructed a standardized corpus of 50 semantically similar sentences across English (en), Spanish (es), and French (fr). Each sentence was designed to test specific vocal parameters: declarative statements, interrogative forms, and sentences with implied emotional weight (e.g., excitement, concern, neutrality). The same voice clone (my own, created via their high-quality English voice cloning) was used for all generations, with all non-language parameters (stability, clarity, similarity boost) held constant via the API.
**Initial Observations (Qualitative):**
* **Spanish (es):** Indeed, the output was largely convincing. Intonation patterns followed expected Spanish cadence, and phoneme pronunciation for Castilian Spanish was accurate. Emotional variance was detectable, though the range between "excited" and "flat" seemed more compressed than in English.
* **French (fr):** This was the notable outlier. The prosody was consistently monotonous, approaching a robotic delivery. While the pronunciation of individual words was technically correct, the linking (*liaison*) and rhythmic grouping of words were poorly executed, resulting in the "flat" characteristic you observed. Emotional prompts were largely ignored.
**Quantitative Test Setup (API):**
I then moved to a latency and consistency test, using the following configuration for 100 generations per language of a standard sentence.
```python
import time
import statistics
# Pseudocode for test loop
latencies = []
for i in range(100):
start = time.perf_counter()
# API call: identical text, identical voice_id, only 'language_code' changed
response = generate_audio(text=standard_text, voice_id=test_voice, language_code="fr") # or "es", "en"
end = time.perf_counter()
latencies.append(end - start)
print(f"Mean Latency: {statistics.mean(latencies):.3f}s")
print(f"Latency StdDev: {statistics.stdev(latencies):.3f}s")
```
**Preliminary Results:**
* **Latency:** No statistically significant difference between `en`, `es`, and `fr`. The model expansion did not introduce processing overhead.
* **Output Consistency (Subjective):** English outputs were 95% consistent in tone quality. Spanish dropped to ~80%. French outputs showed high consistency but at a lower quality tier, with ~70% of samples exhibiting the same prosodic flaws.
My hypothesis is that the underlying acoustic model or the prosody prediction layers have been unevenly trained or fine-tuned across the new language set. French, in particular, may be suffering from a lack of high-emotion, high-variability training data in that language, causing it to default to a safe, monotone delivery.
I am now designing a more rigorous MUSHRA-like listening test for crowd-sourced evaluation. Has anyone else conducted structured comparisons or encountered similar disparities? Specifically, I am interested in:
* Benchmarks for other languages like German or Japanese.
* Any observed differences in using pre-made voices versus cloned voices in multilingual mode.
* Whether the `style` and `emotion` parameters (in the playground) have any measurable effect in non-English languages.
Without reproducible, controlled tests, we are merely sharing subjective impressions. The data must drive the conclusion.
numbers don't lie
numbers don't lie