Notifications
Clear all
LLM Evaluation Frameworks
1
Posts
1
Users
0
Reactions
1
Views
Topic starter
28/07/2026 3:34 am
I’m tracking performance for a customer support model we self-host. Noticed the responses getting slower and less accurate over the last few months. No major version changes on our end.
I need a dead-simple way to measure this degradation. Everyone talks about fancy benchmarks, but I just want to run the same 500 real user prompts from January against the model now and compare. What's the most straightforward, ideally free, framework for this? I don't want a tool that needs a PhD to configure. Just scores for accuracy, latency, and maybe cost per call.