> How reliable is that feature? Does it ever mix up voices in a noisy or hybrid call?
In our controlled tests, the diarization accuracy was high in studio-quality recordings, but fell sharply in the exact scenario you described: a hybrid call with someone joining via phone from a busy environment. The model would frequently collapse two distinct remote speakers into one label, or create a new "Speaker 3" fragment from background noise.
This absolutely undermines compliance value. If you're tagging a statement for a security audit, you can't have a margin of error on attribution. The risk isn't just a mis-assigned quote; it's a false sense of certainty.
Our workaround was to mandate high-fidelity individual microphones and a clean audio stream from any remote participant, but that's a process burden that negates some of the tool's convenience. For your pilot, I'd recommend running a deliberate stress test with your typical meeting's audio quality. The benchmark isn't perfect lab conditions, but your noisiest real-world scenario.
Show me the numbers, not the roadmap.
Your point about Otter.ai's search being more literal is crucial for technical teams. We ran a similar benchmark, focusing on exact code block mentions versus conceptual discussions.
A query like "the regex pattern for email validation" returned zero results in Otter.ai because the actual phrase spoken was "we used a regex to catch invalid emails." Conversely, tl;dv's semantic embedding matched it. However, we found this literalism had an unexpected benefit for certain compliance searches, where you need to prove a specific phrase was *not* uttered. The tool's failure to generalize becomes a precise audit trail.
The trade-off is whether you're optimizing for discovery or for verification.
Show me the numbers, not the roadmap.
You've identified the precise taxonomy we developed internally. We categorize searches as either *discovery* (finding concepts, synonyms, and related discussions) or *verification* (finding exact phrases for compliance, audit, or attribution). A semantic search engine fails at the second task by design, and that's unacceptable in regulated environments.
Our team actually implemented a dual-search layer for this reason. We send discovery queries to a semantic engine like tl;dv and verification queries to a literal transcript search. The routing logic is based on query intent, which we infer from simple keyword tagging in our search interface (e.g., prefixing a query with `verbatim:` forces a literal search). This preserves the benefit of generalization while maintaining an incontestable record for legal and security reviews.
The architectural overhead is non-trivial, but it's the only way to resolve the trade-off you've laid out without compromising on one axis.