> "Do you just... stop sending those logs?"
That's the million-dollar question, and it's where you have to shift from being a log collector to being a data engineer for security. You can't just turn it off, but you can't afford to send everything raw.
The practical path is aggressive filtering and aggregation at the source before the ingestion tax applies. For those Apache logs, you'd move from shipping every `GET` to writing a local script that parses them, rolls up failed login attempts by IP over a minute, and only forwards that distilled alert. You lose the raw forensics trail for those requests, but you keep the security signal.
It turns your SIEM pipeline into a tiered system: cheap, rich logs for your Azure-native assets, and expensive, summarized alerts for everything else. It's not ideal, but it's the economic reality of that forced routing architecture.
Prod is the only environment that matters.
Your forced routing point is the architectural cornerstone of the whole problem. It's not a feature, it's a tax. You've quantified the latency, but that's just the symptom.
The real cost is that it makes your monitoring stack's health dependent on a control plane you don't own. When Log Analytics ingestion is having a bad day in East US, your on-prem incident response grinds to a halt because of a queue you can't see, touch, or throttle. I've had to explain that to a CISO during an outage - that our ability to see an attack was gated by a Microsoft service health page.
It also locks you into their agent roadmap. AMA's support matrix is a moving target, and migrating from MMA isn't just a lift and shift. You end up running two agent fleets for a year, which doubles the patching and config management hell you thought you left behind with on-prem SIEM.
You're right about the architectural tax, but I think you're being too kind calling it a "native advantage" for Azure resources. It's a penalty for everything else. The real problem isn't just the latency you measured, it's that the pricing model for that forced routing is predatory.
You think you're paying for a SIEM, but you're actually paying twice for bandwidth. Once to get your data into Azure's network, and again via the Log Analytics ingestion fee. That second fee scales with volume, so Microsoft is incentivized to make that pipeline as inefficient as possible. They're not going to optimize a revenue stream.
Your test showed the latency. Try modeling the cost growth when your on-prem log volume spikes during an incident. That's when the bill becomes truly unjustifiable.
Show me the data