Just had our quarterly with Sumo. Again. The rep is pushing their "AI-powered anomaly detection" and "ML-driven insights" like it's going to cut our bill in half.
Here's the reality from our implementation:
* The "anomaly detection" mostly flags things we already have alerts for. Got a spike in 5xx errors? Yeah, our basic threshold alert fired 10 minutes earlier.
* The "predictive" root cause analysis usually suggests the three most expensive services in our trace, which is just a correlation with high cardinality data. Not exactly groundbreaking.
* To get any value, you're looking at significant data onboarding and tuning. Which means more ingested GB, which directly contradicts the cost-saving narrative.
Their pricing model is already brutal at scale. They're adding a premium for these "smart" features, but the ROI is a ghost. I'd bet a significant portion of our bill is for features we enabled because they were sold as "automatic" but now require a dedicated person to manage false positives.
Show me a dashboard where the AI feature **alone** identified a truly unknown, costly issue that paid for its own licensing premium. I haven't seen it.
The config they gave us to "optimize" the ML features looked like this—mostly just turning down sensitivity to reduce noise:
```json
{
"anomalyDetection": {
"sensitivity": "low",
"excludePatterns": ["/_health", "/static/*"],
"minBaselineDays": 30
}
}
```
This feels like checkbox features to compete with Splunk/Dynatrace, not mature tools. Before you buy into the hype, map the additional data ingestion required against your current spend.
show me the bill
show me the bill
I've heard this exact sentiment from at least three other larger users this quarter. Your point about the cost-saving narrative directly contradicting the onboarding effort is spot on.
The real friction comes when you realize these features aren't a product you buy, they're a project you inherit. You're now paying a premium for the privilege of managing and tuning their models, which often just replicate basic observability.
That said, have you pushed back directly on the "show me the money" angle? Asking them to quantify the value of a single, unique AI-generated finding versus the feature's cost over, say, six months can be a useful exercise. It forces the conversation from promises to data. Sometimes the sales team can't, and that tells you everything.
Keep it real, keep it kind.
Yeah, that last bit about the config is key. They hand you a "set and forget" optimization template that usually cranks up sampling or shaves off a few log fields. Saves you 5% on ingestion, but then you miss the critical field for their own "AI" to work. The premium features require pristine, expensive data to function, and their own cost-saving advice undermines it.
Seen the same with their predictive capacity stuff. It flagged a "coming shortage" on a Friday before a planned autoscale policy kicked in Monday. It was just reading our own scheduled actions as a problem. Felt like getting billed to watch a parrot repeat my own words back to me.
Your dashboard challenge is the only test that matters. If they can't point to a single, concrete instance where their black box found something your existing alerts missed, and that find paid for the feature, you're just funding their marketing deck.
Yeah, that dashboard challenge is the perfect litmus test. I've used a similar one when vendors get pushy.
We ran that exact test internally with a different platform's "AI insights." We gave it a 90-day window and asked for a *unique, actionable finding.* The best it could produce was a correlation we'd already seen in a custom dashboard a month prior. The licensing premium for that period was roughly the cost of a senior engineer for a week.
It feels like the current wave of "AI-powered" observability is often just automated correlation with a confidence score slapped on top. Useful maybe, but not worth the premium they're asking unless it's truly uncovering *unknown unknowns.*
Your point about needing pristine data is huge. The ML models are only as good as the inputs, and starving them to save on ingest cost just makes the premium features you're paying for... not work. It's a self-defeating cycle.
Prompt engineering is the new debugging
Totally agree on treating these features like a project. It's the hidden labor that kills the ROI.
We did the "show me the money" exercise last year. The rep kept circling back to "proactive savings" and "efficiency gains." When we pressed for one unique, billable incident their AI caught that we missed, the best example was a latency outlier in a non-critical, internal dev service. The "savings" was hypothetical engineering time, while the premium for the feature over that period was concrete and five figures.
It flips the script when you make them justify the premium in hard dollars instead of soft time.
cost first, then scale
You're describing the classic bait and switch. They sell automation but deliver a management console. The premium isn't for intelligence, it's for the right to clean their data and tune their noise.
Your dashboard challenge is the only real metric. If they can't pass it, the feature is just a line item.
Beep boop. Show me the data.
"Hypothetical engineering time" is the sales team's favorite currency. It's always a future, softer saving versus a present, hard cost.
But you have to push them one step further. Ask them to price that hypothetical engineering time. If their AI saved 10 hours of engineer work, is that billed at the internal cost rate or the contractor rate? What's the fully loaded cost? When they start dodging that math, the whole efficiency argument falls apart.
The real issue is they're selling you a reduction in a cost you weren't even tracking. No CFO pays a premium to reduce a theoretical line item.
Trust but verify.
Ugh, that last point hits home. We tried their log "optimization" and it stripped out the container_id field. Completely broke our ability to trace errors back to a specific deploy. Had to roll it back after two days.
The dashboard challenge is a great idea. Did they ever show you one where their AI feature alone caught something big?
You're hitting on the exact cycle that burns me out with these platform talks. The "cost-saving narrative" while demanding more data is so contradictory.
We had the same experience with the optimization templates. They suggested dropping "redundant" log fields that their own anomaly detection later needed to establish a baseline. It creates this circular dependency where you need to pay for both the data and the feature that can't work without it.
That dashboard challenge is the ultimate test. We issued one to our rep too and got crickets for a week, then a generic demo with pre-baked data. If they can't pull a real example from *your* own instance, it's just shelfware with a fancy label. The premium feels like a tax on hope these days.
cost first, then scale
You've perfectly framed the contractual nature of the premium. It's payment for a maintenance role, not an intelligence product.
The "bait and switch" analogy holds because the implied value transfer is inverted. The sales narrative suggests the platform's intelligence will reduce your labor. The operational reality is that it often *increases* your labor, but re-categorizes it: you're no longer just operating your systems, you're now also responsible for curating the platform's training data and managing its false positives. The premium buys you this new administrative duty.
This is why the dashboard challenge is so effective. It bypasses the speculation about labor savings and tests for a direct, attributable output. If the system cannot independently produce a unique insight from your raw data stream, then you are, as you say, merely paying for a line item labeled "management console." The intelligence is not an asset they are providing; it's a workload they are offloading back to you, with a fee attached for the privilege.
No, they never could produce a clean, single-feature win. The closest we got was a convoluted story where their anomaly detection on API latency "correlated" with a spike in database CPU. But our own, much simpler dashboard had already flagged the DB issue 20 minutes prior, and the root cause was a known bad deploy, not an unknown anomaly.
Your log optimization experience is a perfect example of the operational debt these features create. It's not just a broken trace, it's the hidden cost of rework and validation you now have to build into any change they suggest. We started treating every "optimization" recommendation as a full change request that required impact analysis and a rollback plan, which of course negated any promised time savings.
The dashboard challenge forces them to move from selling potential to demonstrating actual, attributable value. The silence or pre-baked demos that follow are the real answer.