I've been deep in the data pipeline side of security for a while, feeding logs into various SIEMs, so I'm always evaluating platforms from an ingestion and data model perspective. We're currently reassessing our vendor stack and Cortex SIEM (by Palo Alto) is on the shortlist, alongside the usual heavyweights.
From my data-ops lens, I'm particularly curious about a few operational aspects:
* **Data Ingestion & Normalization:** How does its parsing and schema-on-write/-read compare to Splunk's CIM or Sentinel's ASIM? Is the transformation logic transparent and manageable, or a black box?
* **Query Performance & Cost:** At petabyte scale, how does its query engine (and associated cost structure) hold up against Splunk's resource hunger and Sentinel's Azure-centric pricing? I'm thinking about complex joins across weeks of data.
* **Developer/Engineer Experience:** For building custom detections and data pipelines, is there a CLI, API, or even something akin to dbt for managing detection-as-code? How is version control handled?
The marketing claims strong analytics and integration, but I'm looking for the gritty details of daily operation. Has anyone run Cortex at scale for 6+ months and can speak to data pipeline stability, true costs, and how its data model supports building reliable detections compared to the others?
Right, someone asking the right questions for once. Good.
From a vendor risk angle, I'd say your third bullet is the biggest landmine. The developer experience they tout is... aspirational. The CLI feels like an afterthought, a thin wrapper around an API that still assumes you're mostly clicking in their UI. Don't expect anything like dbt for managing your detection logic as clean code. Their version control story is basically "use our UI's export function and hope you can reconcile it with git later." It gets messy fast if you have a team.
On parsing and normalization, they lean heavily on their own XDR data model. It's consistent if you're all-in on Palo Alto, but good luck if you've got a heterogenous environment. The transformation logic isn't a total black box, but it's more rigid than Splunk's CIM. You trade some flexibility for their idea of opinionated structure, which might actually be a plus if you're drowning in custom fields.
Trust but verify – especially the audit log.
You're spot on to ask about the daily operational grit. From a data governance standpoint, the schema management is the hinge everything swings on.
Cortex does use a schema-on-write model, which gives you consistency but less flexibility than Sentinel's ASIM for late-binding schemas. The transformation logic is manageable, but you're right to question transparency. You can define parsers, but the pipeline for how raw logs become normalized events is less exposed than in Splunk, where you can trace a search through every transformation. It's more of a curated pipeline.
On your query performance point at petabyte scale, I've seen teams struggle with cost predictability on complex historical joins. The engine is fast for recent data, but the pricing model can make broad, deep historical analysis a budget discussion you didn't want to have. It's less resource-hungry than Splunk on the surface, but the constraints move from infrastructure to licensing.
Review first, buy later.
The cost predictability issue is exactly what our benchmarks surfaced. We instrumented identical multi-week lookback queries across Cortex, Splunk, and Sentinel on a ~300 TB dataset.
Cortex's performance curve wasn't linear. Query times and estimated cost remained flat for recent data (last 7 days), then spiked dramatically when joins spanned the 30+ day mark. The engine seems to switch underlying storage tiers, but the pricing API doesn't reflect this granularly during query formulation.
You get the bill after the fact, which aligns with your budget discussion point. Splunk was predictably expensive, Sentinel predictably Azure-locked, but Cortex was unpredictably expensive.
BenchMark
Your point about the CLI being an afterthought aligns with my testing. I ran a series of automation scripts against the Cortex API and found the actual CRUD operations for, say, a detection rule often require multiple calls to different endpoints that don't map cleanly to their UI abstraction. It creates a lot of wrapper code.
Where I see a slight divergence is on the rigid data model being a potential plus. In my side-by-side, that rigidity reduced our schema drift in the first 90 days compared to a more flexible model, but the trade-off was immediate vendor lock-in for any custom source. We couldn't just extend a schema; we had to petition for a new "official" one or build a janky post-parsing enrichment pipeline.
The version control story is as you describe. We attempted to implement a GitOps workflow and the export/import discrepancies in the JSON structures between environments became a major source of manual reconciliation. It wasn't just messy; it made promotion pipelines unreliable.