Okay, so I finally got around to testing Otter's new AI Chat feature on a few recorded sales calls. Honestly? I'm a bit underwhelmed.
The promise is great: ask questions about the conversation and get instant answers. But in practice, I asked it "What were the prospect's main pain points?" and it just regurgitated a random sentence from the middle of the transcript that mentioned the word "problem." It completely missed the actual, repeated concerns the prospect voiced about integration time and budget constraints. It felt like a keyword search, not an understanding of the conversation flow.
For a marketing ops person, this is a critical gap. If I want to use this to quickly qualify leads or pull out follow-up themes for nurture campaigns, it needs to grasp context. Right now, I can't trust it to accurately summarize objections or next steps, which are the bread and butter of turning calls into actionable CRM data.
Has anyone else tried using it for post-call analysis? Did you find workarounds, or are you also just sticking to manually skimming the transcript? I'm hoping this is just a first-iteration issue they'll improve.
Automate all the things.
Yeah, that's exactly the kind of thing that worries me. I tried asking it "What did we agree to for next steps?" after a support call, and it pulled out a line where I said "I'll have to check on that," which wasn't the final action item at all. It seems to just latch onto phrases that match keywords.
If it's missing repeated themes like budget or integration time, that's a huge problem for actually using the output. It makes me wonder, is there a way to "train" it on what a good answer looks like, or are we just stuck hoping the next update is smarter?
For now, I'm back to the old ctrl+F on the transcript too, which kinda defeats the purpose.
Yeah, that's exactly what I'm worried about. If it's just keyword matching, it's not much better than search.
But I have to ask, were those repeated concerns about integration time spread out? I wonder if the tool struggles with piecing together points from different parts of the conversation, especially if the wording changes each time. It might need the exact same phrase repeated.
I'm curious, did you try asking it a much more specific question, like "What did the prospect say about budget?" Does that force a better answer, or does it still fail?
learning every day
Tried the specific question angle. It just pulls the one quote where "budget" was said verbatim, even if it's an offhand comment. Misses all the context, the subtext, the "we're looking at Q3" replies. It's search, dressed up.
You're right, it can't synthesize. It finds strings. Which makes the whole "AI" label a bit of a joke. You're paying a premium for regex.
Your favorite tool is probably overpriced.
I totally feel that frustration. The gap between the marketing promise and the actual output is real. You nailed it with "turning calls into actionable CRM data" - that's the exact use case we were excited about on my team too.
We tried something similar for product feedback calls. Asking "What were the top feature requests?" just pulled random quotes with "wish" or "want" in them, completely missing the nuanced, repeated asks about bulk editing. Like your budget constraint example, it grabs the literal phrase but loses the strategic context.
It makes me wonder if the current version is only trained on simple transcript Q&A, not on synthesizing business conversations. For now, we're tagging calls manually in Mixpanel with specific keywords before we even run the AI chat, which is an extra step but helps a bit. Have you found any other tricks to get more reliable insights out of it, or are we all just waiting for a model update?
keep building