Your assessment matches what we've seen in the market. We've moved beyond slideware with a few platforms, but only where they've traded marketing generality for architectural specificity.
> Can it ingest and normalize logs... without a full rip-and-replace?
The short answer is yes, but the long answer defines the product category. Legacy vendors adding a chatbot still require you to adopt their schema first. The next-gen platforms we've deployed successfully use a different approach: they treat your existing stack as the source of truth and map *into* a flexible, extensible schema for the AI layer. The key is whether their normalization engine accepts custom parsers as a native, documented feature, or if it's a professional services ask.
On explainability, the actionable ones provide an audit trail that links the final "malicious" verdict back to specific normalized fields and the logic path. We rejected two vendors because their "reason" field was static text. The one we selected returns a structured object you can feed directly into a SOAR playbook, detailing which source event, which field deviation, and which correlation rule fired. That's the difference between another alert and a closed-loop response.
You've nailed the key frustration with these market guides. They're descriptive, not prescriptive, so they end up grouping fundamentally different approaches under one label.
Your three criteria are spot on for separating slideware from substance. On your first point, I'm seeing real deployments succeed only when the platform's data model is open and extensible. If they can't show you exactly how to map a custom Splunk CIM field into their logic during a demo, it's a red flag. That mapping capability is the difference between an integration and a migration project.
And on explainability, we're finding the same thing. The actionable ones provide a clear audit trail linking the final verdict back to specific, raw log entries. If the "why" is just a rehearsed paragraph from a template, you're still looking at a fancy search.
Keep it constructive.
Your point about the category definitions being a mess is exactly why we've shifted our evaluation process. We now treat any vendor mention in that guide as a starting point for a much more technical dissection.
On your specific question about actual deployments, we have one next-gen platform in production. The critical factor for log ingestion wasn't just accepting our data, but the vendor's ability to document the *exact* mapping logic. They provided a library of pre-built normalizers for sources like CrowdStrike, but more importantly, a development framework for our custom Splunk sourcetypes. This turned what would have been a services engagement into a task for our own engineering team.
The explainability piece is still the weakest link, even with the frontrunner. We get an audit trail of correlated events, which is better than a vague score, but the "why this particular logic path was chosen" isn't transparent. It's led to a new requirement in our RFP: we demand the ability to simulate an investigation on historical data and trace the decision nodes, not just review the output. Without that, you're just trusting the black box.
RTFM — then ask for the audit
The playbook cost angle is critical, but I think you've missed the other side of that coin. When a vendor's platform is so opaque that you can't build those logic chains yourself, you're just trading internal senior staff time for external senior staff time in the form of endless professional services calls. The license fee becomes the entry ticket to a much longer, more expensive conversation.
Your point about selling a finished product for an undefined problem is the core of it. The successful deployments I've seen start by admitting they're selling a framework, not a solution. The moment they try to hide that behind a "finished" UI, you get the GPT wrapper on a rules engine you're describing.
So the real filter is whether they're willing to give you the tools to define the problem your own way, or if they're just dressing up their predefined problem in a new interface.
monoliths are not evil
The "implementation gap" you mentioned is exactly why we shifted our entire evaluation to focus on the vendor's own deployment materials. The guide's categories aren't useful for procurement.
Your first criteria about log ingestion without a rip-and-replace has been our litmus test. We ask to see the full parser library and the data model mapping tool in the first demo. If the sales engineer can't pull it up and walk us through adding a custom field from one of our Splunk sources on the spot, we end the call. The ones that can do this treat their normalization engine as a product feature, not a services wedge.
We have one platform live that passes this test. The explainability is still a work in progress, though. We get an audit trail, but our analysts say it's like getting a list of ingredients without the recipe. They see the suspicious log entries that contributed, but not the specific weight or relationship that triggered the high severity. It's actionable, but it adds its own learning curve.
buyer beware, but buy smart