Alright, let's get this out of the way first: the official Sentinel documentation on cost optimization is a masterclass in stating the obvious while avoiding the actual landmines. "Consider your retention periods!" Thanks, I never would have thought of that. After six months of watching our bill climb with our "cloud-native transformation," I decided to stop trusting the Azure Advisor recommendations and actually look at what we were ingesting. The result wasn't just a trim; it was a full-on amputation of dead weight.
The core issue, which Microsoft seems allergic to mentioning directly, is that the default configurations and connectors—especially the Microsoft 365 and Security ones—are designed for the "collect everything, ask questions later" crowd, which is fantastic if your budget is infinite. Our bill was dominated by a handful of tables that provided zero investigative value. The real work wasn't in changing retention (though we did that too), but in surgically stopping the flow of useless data before it ever hit the workspace.
The first step is the only one that requires any real effort: figuring out what you're paying for. The `Usage` table is your friend, but you have to interrogate it properly. Don't just look at total GB per day; you need to see which tables are the culprits, and that requires breaking it down over a meaningful period.
```kusto
Usage
| where TimeGenerated > ago(30d)
| where IsBillable == true
| summarize BillableDataGB = sum(Quantity) / 1024 by DataType, Solution
| render piechart
```
This will paint a damning picture. In our case, `OfficeActivity` was a staggering 40% of our bill, followed by `SecurityEvent` and a bizarrely bloated `AzureDiagnostics` for various resources. The `OfficeActivity` table, in particular, is a firehose of administrative noise from M365. Unless you're actively hunting for a specific user's every click in SharePoint, 95% of this is financial waste.
The pruning process is two-fold: elimination and filtration.
1. **Eliminate Entire Tables:** For some data types, you just need to turn off the tap. We audited every Diagnostic Setting sending logs to the Sentinel workspace. Ninety percent of the VMs sending `SecurityEvent` logs didn't need to; we switched them to a lower-cost, internal-only solution for compliance and kept only critical servers in Sentinel. For `OfficeActivity`, we used the Office 365 audit log policy to stop streaming certain workloads entirely (Goodbye, `SharePointFileOperation` for every department's archive site).
2. **Aggressive Filtering at the Source:** This is where you save the real money, and it's criminal that the Sentinel UI doesn't make this front-and-center. You don't just enable a connector; you must immediately define a custom filter. For the `SecurityEvent` connector on our remaining servers, we implemented a filter at the Log Analytics Agent level (soon to be Azure Monitor Agent) to drop noise events by ID.
```xml
4688
```
We created a collection rule to only collect 4688 (process creation) events with specific command line criteria relevant to our threat model, not every notepad.exe instance. Similarly, for the Azure Diagnostics (`AzureDiagnostics` table) from resources like Key Vault or NSGs, we defined granular collection rules in the Diagnostic Settings themselves, collecting only error-level logs or specific operation names.
The final, often-overlooked, step is to challenge every analytics rule and hunting query. Each one that runs a `union *` or scans `OfficeActivity` without a time-bound, targeted where clause is literally burning money. We refactored our rules to query specific tables and added tighter time filters.
The outcome? A 60% reduction in monthly ingestion costs. Not from the gentle, suggested "adjust your retention from 90 to 30 days," but from the brutal, manual work of turning off the useless spigots. The platform wants to consume; your job as an engineer is to force-feed it only what it needs. The savings will directly fund the actual security engineering work you should be doing instead of babysitting a billing dashboard.
-- Cam
Trust but verify.