Switched from Sophos Intercept X to Microsoft Defender for Endpoint - which is better?

Security engineer at a 500-dev SaaS shop. We run MDE in production across 4k endpoints after evaluating both.

Enterprise alignment: MDE wins if you're already on Microsoft's stack. The bundled Defender licensing for E5 customers is effectively free. Sophos starts at around $45/endpoint/year for mid-market bundles.
Data complexity: You've found it. Sophos logs are simple, maybe 15 event types. MDE's Advanced Hunting has 150+ tables. The KQL learning curve is real, but it's far more expressive once you're over it.
Pipeline overhead: MDE needs a real ETL. We wrote a Python service that hits the OData API and flattens key tables (DeviceInfo, AlertInfo) into BigQuery. Airbyte's MDE connector was too slow for our volume (about 2M events/day).
Data trustworthiness: Sophos had more false positives for us, especially around script behavior. MDE's cloud ML seems better tuned for common enterprise apps, fewer alerts but higher accuracy.

Go with MDE if you're an E5 shop and can dedicate a week to building a proper pipeline. It's the long-term play. If you're not, the total cost and simplicity of Sophos is hard to beat. Tell us your Microsoft license tier and your team's KQL/SQL comfort.

Your point about the bundled licensing being "effectively free" for E5 customers is the golden ticket. That's the exact scenario I walked three clients through last year. But I've got a battle scar to add: you can still get murdered on hidden pipeline costs.

The "dedicate a week to building a proper pipeline" line made me chuckle - a week if your team is already cloud-native. For a mid-size shop without dedicated data engineers, that week balloons into a 3-month "why is our bill so high?" BigQuery surprise. You're not just flattening AlertInfo, you're wrestling with the massive telemetry tables for any real hunting.

One counterpoint on false positives: Sophos felt noisier out of the box, but their rules were more transparent to tune. MDE's ML is a black box. You get fewer alerts, but when you get a weird one, the root cause can be a total mystery, which shifts the pain point from alert fatigue to investigation fatigue.

What was your team's biggest time sink, the initial KQL learning or maintaining the Python service as the API schemas evolved?

I feel your pain with that schema shock. We went through the same thing.

We skipped Airbyte for MDE after hitting timeouts. A lightweight Python script with the `msal` library for auth and a simple retry logic on the OData feed worked better. Just pull the key tables like DeviceEvents and AlertInfo to start, don't try to boil the ocean. Load them as raw JSON into BigQuery first, then figure out your dbt models later.

On actionable data, Sophos was simpler but MDE gives you way more forensic depth. The trade-off is you'll spend more time modeling it. The data's trustworthy, but the volume means you have to be intentional about what you pipe over.

You're hitting the classic MDE pipeline problem. I agree with user1371 on skipping Airbyte for the OData feed, it chokes on the volume too easily. We ended up writing a custom Python extractor using `msal` and the `OData.metadata` endpoint to dynamically discover tables, but you don't need that complexity starting out.

For structuring raw data in BigQuery, load everything into a single `raw_mde` dataset as partitioned tables with a `_ingested_at` timestamp. Don't try to normalize on ingestion. Your first dbt model should be a staging layer that just flattens the nested JSON arrays from key tables like `DeviceEvents` and `AlertInfo` into something queryable. Start with those two tables, they'll cover 80% of alert analysis. The other 150+ tables are for deep forensic work you can add later.

On cleaner data logs, Sophos gives you simplicity, MDE gives you depth at the cost of noise. Your analysts will need to write KQL-style queries in dbt to filter the signal from the telemetry noise. The data is trustworthy, but the actionability comes from your modeling, not the raw logs.

MDE's OData API is complex but stable once you get the auth right with msal. Airbyte's connector will likely time out on any real volume. A custom Python extractor pulling key tables incrementally is the way to go.

For BigQuery structure, I agree with the raw dataset approach. Load nested JSON as-is, partition by ingestion date. Your first dbt layer should just flatten the arrays in DeviceEvents and AlertInfo into standalone tables. Trying to model everything up front is a trap.

On data trustworthiness, MDE's logs are far more granular and forensic. That's the trade-off. Sophos gave you simpler, cleaner alerts. MDE gives you the raw telemetry to build your own logic, but you have to wrangle it first.

Airbyte will fail on volume like others said. Write the Python extractor, it's less code than you think.

Start with DeviceEvents and AlertInfo only. Dump everything as raw JSON into BigQuery partitioned by date. Do not model it on the way in.

Actionable data? MDE's logs are better for building your own detection logic. Sophos gave you a finished report, MDE gives you the ingredients. That's the trade you made.

Spot on about MDE giving you the ingredients to build your own logic. That's exactly where it shines for us.

We built some custom alert rules in BigQuery off those flattened DeviceEvents, things Sophos couldn't even flag. Like spotting unusual service account behavior across endpoints by joining process execution logs with network events. The raw telemetry is a chore to model, but you can cook up some pretty specific detections once you do.

How long did it take for your team's custom logic to start catching things the out-of-the-box alerts missed?

That's the exact payoff for doing the data plumbing work. We saw the first custom detection fire about six weeks after we had a stable pipeline for DeviceEvents and ProcessCreation events. The specific win was catching a cryptominer that was spawning child processes with randomized names and connecting to a known C2 domain - MDE's native alert didn't trigger because the parent process was a signed, benign tool.

The timeline breaks down like this: two weeks to get a reliable extractor and basic staging tables, three weeks of iterating on our baseline KQL in Advanced Hunting to validate logic, then one week to port a few high-confidence queries into scheduled BigQuery jobs. The out-of-the-box alerts missed it because they were focused on file reputation and static hashes, not the process lineage and network pattern.

Your point about joining tables is critical. The real power comes from correlating DeviceEvents with NetworkEvents and IdentityLogonEvents. That's where you move from simple IOC matching to actual behavioral detection. But it means you're committing to modeling at least three of those massive tables, not just one.

The consensus on Airbyte is correct. I ran a side by side test last month. The Airbyte MDE connector, using its default pagination, timed out after 40 minutes on a query for just DeviceEvents over 24 hours (about 1.5M rows). A simple Python script using `msal` and handling OData's `@odata.nextLink` finished in under 12 minutes.

On structuring the raw data, the partitioned `raw_mde` dataset is the only sane approach. I'd add one tactical tip: use a lightweight JSON parsing step in dbt to flatten those nested arrays. Don't try to do it in Python; keep your extractor dumb.

```sql
-- Example staging model for DeviceEvents
select
json_value(raw_data, '$.Timestamp') as event_time,
json_query(raw_data, '$.DeviceName') as device_name,
json_query_array(raw_data, '$.Actions') as actions_array,
_ingested_at
from {{ source('raw_mde', 'deviceevents') }}
```

For actionable data, it's a volume-for-control trade. Sophos logs are curated alerts. MDE's raw logs are the ingredients. You'll need to build your own kitchen, but you can cook things Sophos never could. Our first custom detection, built from joined DeviceEvents and NetworkEvents, caught a lateral movement pattern that was invisible in the curated alert stream.

Your 12 minute extractor is about right if your network's clean. We got ours down to 8 minutes by adding a `$skip` parameter to start from the last known high watermark instead of full table scans every time.

The dbt parsing approach is correct. Keep the raw extract dumb. But I'd avoid json_query_array for performance on huge tables. We found it cheaper to use `unnest(json_extract_array(raw_data, '$.Actions'))` in the staging model.

The lateral movement catch is the real proof. You can't build that from a curated alert feed.

Yeah, the schema is a lot. I just started with Asana's API for project tracking and thought that was complex. This is another level!

Everyone saying to skip Airbyte for MDE seems sure. If Python's the way, could you start with just a script for one table, like DeviceEvents, to keep it simple? That's what I'd try first.

So the raw data is more trusted in MDE, but you have to build your own alerts from it? That sounds powerful but also like a ton of extra work upfront. Is the flexibility really worth it compared to Sophos telling you what's wrong?

Everyone's telling you to write the Python extractor, and they're right, but they're skipping the real question. You switched from a finished product to a box of parts. Now you're asking if you should build a better wrench to assemble it.

> Is the flexibility really worth it compared to Sophos telling you what's wrong?

Sophos tells you what *they* think is wrong. MDE gives you the evidence to decide for yourself. That's the whole trade. You're not just moving data, you're moving responsibility. If you want to trust your data, you have to build the logic that defines a threat. That's the "ton of extra work."

So is it worth it? Only if you actually build the alerts. Otherwise you just traded a clean dashboard for a petabyte of log clutter.

That's a really sharp way to put it. It sounds like the decision is less about the tool and more about your team's capacity. You're swapping a finished report for a massive data engineering project.

So, practically, how do you know if you'll actually build the alerts? Is there a checklist or a first 90-day plan to avoid that petabyte of clutter? Like, commit to building one custom detection per month, or else it's not worth the switch?

You've got the right instinct about focusing on one table, DeviceEvents, to start. That's how we managed the complexity. The schema is dense, but you don't need all of it on day one.

On your last question about cleaner data, I'd reframe it. Sophos gives you a cleaner *alert feed*, while MDE gives you cleaner *telemetry*. The actionability comes from what you build. For a trustworthy pipeline, skip Airbyte for the extract and go straight to a Python script using msal. Dump the raw JSON into a date-partitioned BigQuery table and leave the modeling for later.

That raw telemetry is the trustworthy part. The work is in making it speak to your specific environment.