Hey everyone,
I've been hearing a lot of buzz about AI meeting assistants like Sembly, and as someone who lives in CRMs and automation platforms, the promise is huge. But I'm always skeptical about the "accuracy" claims. To really see how it stacks up, I decided to run a practical, real-world test: pitting Sembly against a skilled human note-taker across 10 actual internal and client meetings.
**The Setup:**
* **Meetings:** Mix of 4 internal project syncs (fast-paced, technical jargon), 3 client discovery calls (relationship-focused, nuanced questions), and 3 workshop-style brainstorming sessions (chaotic, overlapping voices).
* **Human Baseline:** A colleague known for meticulous notes acted as our control. They used a simple text editor.
* **Sembly:** Used the standard plan with automatic recording/transcription enabled. I fed it the same meeting audio files (from Zoom/Teams recordings) that the human used.
* **Evaluation:** I compared the outputs across three key dimensions: **Transcription Accuracy, Action Item Extraction, and Overall Summary Quality.**
**Here’s what I found, broken down:**
**The Good (Really Impressive):**
* **Raw Transcription Speed & Completeness:** This is Sembly's undeniable strength. Having a full verbatim transcript in minutes is a game-changer for searchability. The human simply can't compete on volume.
* **Keyword & Topic Tagging:** Sembly automatically identified project names, tools mentioned (like "Zapier" or "HubSpot"), and deadlines. This is pure gold for later tagging in a CRM or project management system. The human did this manually, post-meeting.
* **Consistency:** The human note-taker had an off day in one meeting and missed a few key points. Sembly's performance was uniform across all 10 meetings.
**The Not-So-Good (Where the Human Won):**
* **Nuance & Intent:** In two client meetings, a client said, "That timeline might be ambitious." Sembly transcribed it literally. The human note-taker added the context: "[Client expressed concern about deadline, may need to revisit.]" This subtlety is critical for CRM notes.
* **Action Items in Chaos:** During the brainstorming sessions, action items were often half-formed ideas ("Someone should check the API limit on that."). Sembly frequently missed these or attributed them to the wrong person. The human, understanding the context, could capture "**Ian** to investigate Stripe API rate limits."
* **Technical Jargon & Acronyms:** For our internal tech syncs, Sembly butchered some proprietary tool names and acronyms (e.g., "CLM" became "claim"). The human, being domain-experienced, got them right.
**My Integration-Focused Takeaway:**
Sembly isn't a replacement for a human note-taker in complex discussions. It's a powerful **force multiplier**. The workflow that worked best for me was:
1. Use Sembly as the primary, unfiltered capture tool. Get that full transcript and its auto-generated topics.
2. **Integrate that output into your workflow.** I set up a Make (formerly Integromat) scenario that takes Sembly's summary (via webhook) and creates a draft in Coda, tagging it with Sembly's auto-detected keywords.
3. Have a human (the meeting owner) spend 3 minutes reviewing the action items and summary, adding the crucial layer of nuance and correction *before* that data syncs to the CRM or project management tool.
This hybrid approach gives you the speed and structure of AI with the contextual intelligence of a human. For straightforward, informational meetings, Sembly alone might suffice. For anything involving sales, sensitive feedback, or complex problem-solving, you still need that human-in-the-loop—but Sembly cuts their work in half.
Would love to hear if others have tried similar tests or built automations around the Sembly output.
api first
api first