Hi everyone. I'm new to the community but have been lurking for a while, absorbing the fantastic discussions here. I'm currently in the middle of a major procurement project for our legal department, specifically evaluating AI-powered transcription services for depositions. As part of our due diligence, I ran a detailed benchmark that I thought this community might find useful.
We took a 45-minute segment of a past deposition (with sensitive data redacted, of course) and processed it through three channels:
* A seasoned, professional human transcriptionist (our current, expensive standard).
* Otter.ai's Business plan, using its automated service.
* Otter.ai's Business plan, but with a human editor from their add-on "Otter Pilot" service applied afterward.
My focus was on total cost of ownership, accuracy in a complex domain, and the implementation/editorial overhead. Here are my granular findings:
**Accuracy & Context (Word Error Rate - WER):**
* **Human Professional:** Near-perfect. Handled legal jargon, cross-talk, and poor audio from a remote participant flawlessly. The baseline.
* **Otter.ai (Automated):** Surprisingly good for general flow, but stumbled in critical areas. It mangled specific case-related terms (e.g., "Novation" became "motion"), struggled with speaker identification during rapid exchanges, and completely missed some low-volume utterances. Estimated WER around 8-12%, but the errors were in high-importance words.
* **Otter.ai + Pilot Editor:** Accuracy improved dramatically, getting very close to the human transcriptionist. The editor caught and corrected the legal terminology. However, this adds time and cost.
**Workflow & Time Investment:**
* **Human Professional:** 24-hour turnaround. Our internal overhead is near zero—just sending the file and receiving the transcript.
* **Otter.ai (Automated):** ~10 minutes for the initial transcript. But then, a senior paralegal spent **over an hour** proofreading and correcting, which is expensive internal labor.
* **Otter.ai + Pilot Editor:** ~10 minutes for automation, plus a stated 24-hour turnaround for the edit. We received it in about 30 hours. Internal review time dropped to just 15 minutes for a spot-check.
**Cost Analysis (for this 45-minute file):**
* Human Professional: Fixed fee of $180.
* Otter.ai (Automated): Direct cost is just the monthly seat (~$30/user/month prorated), but the **hidden cost** was ~$75 of paralegal time for correction.
* Otter.ai + Pilot Editor: Monthly seat cost + $40 add-on for the Pilot edit. Minimal internal review.
This leads me to my main questions for the community, especially those who've implemented Otter.ai in similar regulated or technical fields:
1. **Long-term TCO:** Has anyone done a full TCO model comparing pure human transcription against a hybrid AI+human editor model like Otter Pilot? I'm concerned about scaling the Pilot editor costs and potential variability.
2. **Implementation & Training:** Was there a significant onboarding period to "train" the software on your specific jargon (e.g., unique product names, internal acronyms)? How did you manage that?
3. **Exit Strategy:** Otter.ai seems to have its own ecosystem. How portable are the transcripts if we decided to switch vendors in 18 months? Is the data easily exportable in a format another service could use?
4. **User Reviews from Legal Teams:** Beyond accuracy, are the attorneys and paralegals actually *happy* with the workflow? Do they trust the output enough to reduce their own proofreading burden, or does it just add another step they resent?
I'm leaning towards a pilot program with Otter's Business + Pilot plan, but I want to be thorough. The potential savings are significant, but the risk of errors in a legal context gives me pause. Any insights or experiences you can share would be incredibly valuable.