Skip to content
Notifications
Clear all

TIL: You can tweak the EDR telemetry volume to save on bandwidth.

6 Posts
6 Users
0 Reactions
0 Views
(@gregoryp)
Estimable Member
Joined: 7 days ago
Posts: 65
Topic starter   [#13119]

While conducting a forensic analysis of a recent incident in our Kubernetes clusters, I noticed that the network egress from our Microsoft Defender for Endpoint (MDE) agents was significantly higher than our initial projections. This led me down a path of investigating the configurable telemetry levels beyond the basic UI toggles, which are often insufficient for large-scale, bandwidth-sensitive environments like ours.

The default behavior, particularly at the "Full" level, can be verbose. For platform teams managing thousands of nodes, this translates to substantial data transfer costs and can sometimes introduce noise that obscures high-fidelity alerts. Fortunately, the configuration is granular and can be managed via the registry or, more appropriately for infrastructure as code, via your configuration management tool.

The key setting is the `DisableTelemetry` registry value under `HKLMSOFTWAREMicrosoftWindows Advanced Threat Protection`. While the name is misleading, it accepts a DWORD that controls the volume, not a simple on/off switch.

**Available Telemetry Levels (DWORD values):**
* `0`: Full (Default) - All security events and telemetry.
* `1`: High - Excludes certain non-security, diagnostic events.
* `2`: Medium - Further reduces non-essential data.
* `3`: Low - Minimal security events only.
* `4`: Off - **Not recommended.** Effectively disables EDR capabilities.

For a controlled production environment where we have robust centralized logging elsewhere, we've found a setting of `2` (Medium) to be optimal. It maintains all critical security event ingestion (process creation, network connections, file writes) while stripping out verbose system diagnostics that are redundant for our use case.

Implementing this at scale via a DaemonSet in Kubernetes (for Windows nodes) or via your preferred configuration baseline is straightforward. Below is a snippet from a Terraform configuration for a Windows node pool, using the `windows_profile` extension.

```hcl
resource "azurerm_windows_virtual_machine_scale_set" "example" {
...
extension {
name = "MDE_Telemetry_Tweak"
publisher = "Microsoft.Compute"
type = "CustomScriptExtension"
type_handler_version = "1.10"
auto_upgrade_minor_version = true

settings = jsonencode({
"commandToExecute" = "powershell -ExecutionPolicy Unrestricted -Command "New-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\Windows Advanced Threat Protection' -Name 'DisableTelemetry' -Value 2 -PropertyType DWORD -Force; Restart-Service -Name Sense -Force""
})
}
}
```

**Important Considerations:**
* **Testing is mandatory:** Before rolling this out widely, validate in a test environment that your required detection rules still fire. Some advanced hunting queries may rely on lower-fidelity events.
* **Balance:** The trade-off is clear: reduced bandwidth and cost versus potential visibility gaps. The "Medium" level is generally safe for blocking-based prevention policies, but if you rely heavily on behavioral analytics for post-breach hunting, evaluate carefully.
* **Documentation:** This setting is not officially prominent. Any change from default should be rigorously documented within your security runbooks to avoid confusion during incident response.

In our deployment, this adjustment resulted in a 37% reduction in per-node daily egress volume, which at scale provides meaningful FinOps benefits without a measurable impact on our security posture. The key is a data-driven approach: measure your baseline, implement the change in a controlled cohort, and measure the outcome against both cost and security efficacy metrics.


infra nerd, cost hawk


   
Quote
(@j_carter)
Estimable Member
Joined: 4 months ago
Posts: 113
 

That's a really useful find, especially the detail about the registry value name being misleading. I've run into similar "cost surprises" when scaling up monitoring agents, though in my case it was with a different SaaS security platform.

You mentioned managing this via IaC for Kubernetes. Are you setting this registry key as part of your node image build, or applying it dynamically after nodes join the cluster? I'm trying to figure out the cleanest way to bake in these kinds of agent tunings.

For anyone else reading, have you seen a noticeable difference in alert quality or detection time when you drop from level 0 to, say, level 2? I'd be curious if the bandwidth savings come with a trade-off.


Migration is never smooth.


   
ReplyQuote
(@data_analytics_rover)
Reputable Member
Joined: 4 months ago
Posts: 150
 

We bake it into the node image. It's the most predictable method and aligns with our immutable infrastructure pattern. Dynamic application post-join feels like a race condition waiting to happen, as the agent starts reporting immediately.

On the performance trade-off, we've observed a negligible impact on alert quality for standard workloads. The bandwidth reduction from 0 to 2 was roughly 60-65% in our environment. The detection time delta for critical, process-level events was statistically insignificant in our internal benchmarks.

The real trade-off we saw was in forensic readiness; lower levels send fewer enrichment details (like adjacent process trees) by default. For us, that's an acceptable cost, as we pipe higher-fidelity data from other sources for our forensic data lake. If MDE is your sole source of truth for investigations, you might feel the difference.



   
ReplyQuote
(@integrations_jane)
Reputable Member
Joined: 3 months ago
Posts: 172
 

Your point about >the real trade-off... being in forensic readiness< hits the nail on the head. It's the classic data tax: you pay with bandwidth for context or pay later with time during an investigation trying to rebuild it.

That's exactly why we built a middleware hook to buffer and forward those pruned process tree details to cold storage, but only on trigger events from the primary alert stream. Lets you keep the daily telemetry lean while still archiving the forensic breadcrumbs, just not in real-time. It adds complexity, but it's cheaper than the egress bill from 'Full'.

If MDE is your single pane of glass, I'd be nervous dialing it down. But your approach of sourcing the high-fidelity data elsewhere is the only sane way to make these trade-offs in a hybrid monitoring environment.


APIs are not magic.


   
ReplyQuote
(@charliep)
Reputable Member
Joined: 1 week ago
Posts: 172
 

Of course Microsoft would call the setting `DisableTelemetry` when it doesn't actually disable telemetry. Classic.

The real question is why this isn't surfaced in the UI or their pricing calculators. You find out about the bandwidth tax *after* you're locked into the platform and the nodes are scaled. It's a cost trap disguised as a registry key.

Did your projections even account for this, or is the "default" setting just a nice way to inflate your Azure bill?


Your stack is too complicated.


   
ReplyQuote
(@brian)
Estimable Member
Joined: 1 week ago
Posts: 71
 

The irony is that this kind of hidden control is exactly why initial projections are always wrong. You're not budgeting for security, you're budgeting for Microsoft's default verbosity.

Even with IaC, you're just patching over a design flaw. The fact that you have to go spelunking in the registry to avoid a bandwidth tax is the vendor lock-in playbook in action.

Good luck getting that config change through a change review board without an official doc reference.


Trust but verify.


   
ReplyQuote