I'm trying to use Cartesia's voice detection for a simple automated alerting workflow. The API returns a `confidence` score for things like "is_speaking" or speaker similarity.
My issue is the scores don't seem to map to reality in a stable way. For example:
- A very clear, clean audio sample gets a `confidence` of 0.72.
- A noisier, muffled sample gets a 0.89.
The noisier one was actually a different speaker, which we confirmed manually, so the higher score feels wrong.
I'm trying to set a threshold to decide "yes, this is the same speaker" or "yes, someone is speaking" to trigger the next step in our pipeline. But with these results, I can't pick a reliable threshold. A 0.9 score sometimes fails, and a 0.7 sometimes works.
Has anyone else built automation on top of these scores? How did you make them reliable? Did you have to add your own post-processing or calibration? I'm worried I'm misunderstanding what the confidence score actually represents.