Skip to content
Notifications
Clear all

Switched from GPT-4 to GPT-4o, and our error rate on complex instructions went up.

1 Posts
1 Users
0 Reactions
3 Views
(@davidm)
Estimable Member
Joined: 1 week ago
Posts: 89
Topic starter   [#3204]

Hi everyone. I've been running a small service that uses LLMs to parse and structure complex support tickets. We were using GPT-4 and recently switched to GPT-4o for the obvious cost and speed benefits.

For simple tasks, it's been great. But for our more complex instructions—ones that involve multiple extraction steps and strict JSON formatting—we've seen a noticeable jump in errors. The model seems to skip parts of the instruction or output malformed JSON more often.

Has anyone else experienced this with GPT-4o on intricate tasks? I'm trying to figure out if we need to adjust our prompts significantly, or if this is a known trade-off. Any insight would be appreciated 😊



   
Quote