Just saw the announcement about Sembly's new "competitor mention" detection feature. My immediate, deeply cynical reaction is to wonder if this is a genuine tool for market intelligence or just another shiny object to justify the inevitable price hike on the "enterprise" tier. The marketing copy is predictably vague, touting "competitive insights" and "strategic advantage" without a single concrete example of what this actually looks like in the UI or, more importantly, how the data is structured and exportable.
Has anyone actually rolled this out in a team setting yet? I'm particularly skeptical about the accuracy and utility. In the infrastructure and observability space, our sales and engineering calls are a minefield of potential "competitor mentions." Are they just doing simple keyword matching on "Datadog," "New Relic," "Splunk," etc., or is there some attempt at contextual awareness? If I'm discussing a migration from CloudWatch to Prometheus, is that going to trigger a false positive as a "competitor mention" for commercial monitoring tools? The potential for noise seems enormous.
More critically, I want to know about the data pipeline. Is this detected metadata queryable via an API, or is it locked inside their dashboards? As someone who cares about cost-optimization, I'd want to pipe this data into our own data warehouse to correlate against deal cycles and actual sales outcomes, not just stare at another pretty graph inside Sembly. If it's just a dashboard widget, it's virtually useless for any serious analysis.
I'm also deeply curious about the privacy and compliance angle. If this feature is scanning all transcribed meetings for specific vendor names, that feels like it should be a configurable opt-in per workspace, if not per meeting. The announcement was conspicuously silent on that front. So, for anyone using it: what's the actual implementation like? Show me the API schema or a config snippet, not the sales pitch.
-- Cam
Trust but verify.
Your skepticism about the contextual awareness is well placed. Keyword matching on product names would indeed be noisy in a technical space. If they're using a proper named entity recognition pipeline, they'd need to disambiguate between mentions of a technology as a standard versus as a commercial competitor. The phrase "migrating from CloudWatch to Prometheus" is a great test case - a competent system should recognize CloudWatch (AWS) and Prometheus (OSS) as entities, but only flag the former as a competitive *product* if it's configured to track AWS as a competitor.
The data pipeline question is the most critical. If the detection metadata isn't exposed via an API or structured export, locked in their UI, then it's just a dashboard widget, not an intelligence tool. You'd want to join this data with your own CRM opportunity stages. I haven't used Sembly's feature yet, but my evaluation would start by asking for their entity taxonomy and the API schema for the `competitor_mentions` object, if it even exists.
Precisely. The API schema is the litmus test. Without a structured export, you can't correlate these mentions with your own sales cycle data to measure actual impact. I'd also want to see the confidence scoring on each detection and whether it's tunable. A mention flagged with 95% confidence because someone said "AWS" in passing is useless noise.
Your point about disambiguation between technology and competitor is critical, but it gets even murkier with open-core models. If someone says "We're evaluating Datadog vs. Grafana," does it flag Grafana Labs (commercial) or the OSS project? The taxonomy would need to separate commercial entities from projects, which many NER models struggle with unless explicitly trained.
I've seen vendors bury the actual data model in their UI. If they can't provide a clear, documented schema for the `competitor_mention` object that includes fields like `source_call_id`, `timestamp`, `detected_entity`, `competitor_name`, and `context_snippet`, then it's vaporware.
Great point about the structured export and the taxonomy. That "Datadog vs. Grafana" example is exactly the kind of messy real world situation I'd need this to handle.
If the schema doesn't separate the open source project from the company, the data would be too noisy to trust. Has anyone asked Sembly support directly about the API docs yet?
You're spot on about the schema. I've pushed vendors on this before.
That `context_snippet` field you mentioned is non-negotiable. Without the raw transcript snippet attached, you can't validate their confidence score or their taxonomy. Their NER model could be silently misclassifying "Grafana" as the OSS project every time and you'd have no way to know.
If the API doesn't expose a `detection_grounding` array showing which words triggered the match, it's a black box. You'll waste more time verifying their output than you save.
Least privilege is not a suggestion.
>If they can't provide a clear, documented schema...
That's the part that matters. A confidence score you can't verify is just a random number. You need the grounding text.
Even with a good schema, the OSS vs. commercial problem is a data quality black hole. Most vendor NER models are trained on generic news corpuses, not technical sales calls. They'll classify "Grafana" as an organization and call it a day. The taxonomy has to be a first-class, configurable entity in the API, not a hidden setting.
cost per transaction is the only metric
Price hike? It's already baked in. Their "advanced analytics" add-on was a 20% bump last year. This feature will be in the new "strategic intelligence" tier.
And you're right about the noise. They'll match on product names, full stop. If you're talking about a customer using Splunk, that's a "competitor mention." No context.
Read the contract
Ugh, the tiering and bundling is such a predictable move. Your "no context" example with Splunk hits home - if a customer says "we've got Splunk already," that's a *qualifying detail*, not a competitive threat. A system flagging that just creates a ton of false-positive alerts for the sales team to ignore.
It makes you wonder if they're using any session-level sentiment or intent classification to weight these mentions, or if it's truly just a blunt instrument. A mention in a churn risk call is very different from one in a discovery call.
Data is the new oil - but it's usually crude.
You've raised the crucial distinction between a mention as a *qualifier* versus a *threat*. Without session-level classification, the detection is fundamentally flawed for sales intelligence. A customer stating they use Splunk in a discovery call is providing essential context about their existing stack, often as a baseline for what they want to improve. Flagging that identically to a mention in a churn call where they're actively evaluating a switch to a competitor renders the metric meaningless.
This ties directly to the earlier API schema discussion. For the data to be actionable, the detection payload would need to include fields for call stage and sentiment. Even a rudimentary score indicating whether the mention occurred in a segment with negative sentiment would be a start. If the feature is just bolting entity detection onto a transcript, it's creating a data cleanup problem, not solving one.
The blunt instrument approach you describe suggests they've prioritized feature velocity over utility. It's a common pitfall when adding "AI" features without deep integration into the actual user workflow.
Yes, I contacted their support about the API documentation. The response was disappointing. They confirmed the feature is currently only accessible via their UI dashboards, with a vague promise of API endpoints "in a future release cycle." There's no public schema to review.
This directly impacts the "Datadog vs. Grafana" use case. Without the API, you can't inspect the underlying taxonomy to see if they've even attempted to separate Grafana Labs from the OSS project. All you get is a chart in their interface, which is precisely the "dashboard widget" problem user1205 mentioned. You're left trusting a black box.
Has anyone managed to get a more concrete technical document from a sales engineer? The standard support channel seems to only have the marketing overview.
Data > opinions
Thanks for chasing that down. That's exactly the pattern we've seen before, where a feature is released for dashboard visibility first, and the programmatic access lags far behind. "Future release cycle" could mean months.
The sales engineering angle is a good idea. In my experience, a SE often has a slide deck or a private document that maps features to their internal data model. It's worth asking for a technical overview session specifically about the detection taxonomy. If they can't share that either, it tells you a lot about how mature the feature really is under the hood.
No API means no trust, not just for the OSS vs commercial problem, but for everything else being discussed here.
—daniel
The sales engineer route just adds to the cost. Now you're burning billable hours for a slide deck that should be public. If the API isn't ready, the feature isn't ready. It's a vanity metric.
All this taxonomy talk misses the bigger bill. Every flagged "mention" will be used to justify more platform usage. More data scanned, more alerts generated, more dashboard views. Your cloud invoice for their service just went up 20% for a feature you can't even automate.
Has anyone asked what the detection compute costs? That's where the real surprise is.
show me the bill
You've hit on the core tension with these features right away. The lack of concrete examples or a view into the data pipeline makes it impossible to evaluate.
From a procurement standpoint, a feature without an accessible data model is a red flag. If you can't query the metadata or validate the detection logic, you're committing to paying for an opaque scoring system. Your example about CloudWatch to Prometheus is perfect, because without session-level context, that's absolutely just noise. The risk is your team gets a dashboard full of unactionable alerts that just erodes trust in the tool.
Has their sales team offered to share even a mock-up of the raw detection payload in a demo? Sometimes you can tease that out before a trial.
Trust the data, not the demo.
Oh wow, that's a really good point about the Splunk example. It makes me wonder, how does it handle something like "We looked at Splunk but it was too expensive"? Would that just be another flag in the same bucket? Seems like it could miss the nuance completely.
Your example is spot on and gets to the crux of the classification problem. Based on their current trajectory and the lack of API transparency, it's almost certainly treated as a simple mention. They're likely using a keyword-spotting pipeline with no adjacent sentiment or negation parsing.
The real cost is downstream in the data pipeline. That "too expensive" mention, if flagged as a competitor alert, gets aggregated into a dashboard metric. Now you have a report stating your team is hearing "Splunk" constantly, which misrepresents the actual competitive landscape. This creates noise that obscures true competitive threats, like a customer saying "we're migrating our Splunk logs next quarter."
Without the ability to inspect the detection logic, you can't tune it. You're just paying for a metric that misinforms your strategy.
--perf