I've been evaluating Le Chat's multilingual capabilities against our existing observability workflows at Datadog, and I'm encountering a significant performance gap with non-English prompts. The documentation states parity across supported languages, but my benchmarking shows measurable latency increases and accuracy degradation when using anything other than English for technical queries.
For example, when asking Le Chat to analyze a Python trace pattern in English, response time averages 1.2 seconds with coherent, actionable suggestions. The same query in French—asking to identify a potential service latency issue from a trace—results in a 2.8-second response with a less precise recommendation that misses a key database span correlation. The trace structure is identical, only the query language differs.
```python
# Example of the type of trace pattern being analyzed
{
"service": "payment-api",
"operation": "process_payment",
"duration_ms": 2450,
"spans": [
{"name": "db.payment_query", "duration_ms": 1200},
{"name": "external.gateway_call", "duration_ms": 800}
]
}
```
This discrepancy becomes problematic when our global teams use Le Chat for troubleshooting. Non-English speaking engineers receive slower, less accurate analysis of their APM data, which contradicts the platform's stated multilingual design. I've tested with Spanish, German, and Japanese prompts with similar results: increased hallucination rates on metric interpretations and a tendency to revert to generic observability advice rather than Datadog-specific insights.
Has anyone else conducted systematic performance comparisons across languages? I'm particularly interested if others have found workarounds for this latency issue, or if there are specific prompt engineering techniques that mitigate the accuracy drop. The vendor's support team acknowledged the difference but called it "within expected variance," which doesn't align with the operational impact we're seeing during incident response scenarios.
null