Everyone's raving about Le Chat's structured output, but try using it for anything beyond a toy example and it's a coin toss. I need to generate consistent, validated JSON for our internal platform's config API. Half the time it's perfect, the other half it's a malformed mess wrapped in extra backticks or a markdown block, or it just hallucinates fields.
I'm invoking it via the API with `response_format={ "type": "json_object" }` and a system prompt that screams "YOU MUST OUTPUT JSON". The schema is defined in the user prompt. Yet, I still get variations. Sometimes it's `json`, sometimes it's a ```json ``` block. Sometimes it adds a cheerful "Here's your JSON:" preamble.
```json
// What I ask for:
{
"service": "edge-router",
"replicas": 2,
"env": [{ "name": "LOG_LEVEL", "value": "info" }]
}
// What I might get:
Here is the configuration you requested:
```json
{
"service": "edge-router",
"replicas": "2",
"env_vars": [{"key": "LOG_LEVEL", "val": "info"}]
}
```
```
Notice the stringified number, the renamed keys, the completely different nested structure. This isn't a minor formatting issue; it breaks our automation.
I've tried the `strict` parameter in my JSON schema definition, but the adherence seems... whimsical. Has anyone actually gotten this to work reliably in a production pipeline, or is this just a feature demo that falls apart under load? What's your actual, working prompt and schema pattern that survives more than ten consecutive calls?