Hey everyone, hoping the hive mind can help me out here. I've been running a custom ingestion pipeline for Sentinel for about 18 months now, pulling in niche application logs from our legacy marketing platforms that don't have a built-in connector. It's been humming along nicely, built mostly on the Data Collector API.
Well, this morning, the entire flow broke. 😬 No data since about 02:00 UTC. My initial script retries just threw back authentication errors, and after some deep diving, I'm pretty sure the API endpoints or required headers have changed. The Microsoft Docs page I was using seems to have been updated quietly, and my old bookmarks are redirecting to newer, more "general" guidance that assumes you're using something like Logic Apps.
My setup was a Python script running in a container, using a service principal for auth and posting directly to the ` https://{workspaceId}.ods.opinsights.azure.com/api/logs?api-version=2016-04-01` endpoint. The error log is pointing to a `401 Unauthorized`, but the SPN's credentials are definitely still valid and have the necessary `Microsoft.OperationalInsights/workspaces/write` permission.
Here's what I've tried so far:
* Re-regenerated the client secret for the service principal (no change).
* Verified the workspace ID and primary key are correct in the DCR mapping (they are).
* Checked the Azure Activity log for the Sentinel workspace – no signs of explicit changes or disruptions from our team.
This has me scrambling because we rely on this data for security event monitoring around our marketing cloud user access. I'm currently rebuilding a test pipeline using a Data Collection Rule (DCR) and a custom Azure Function, as the docs now seem to heavily push, but that's a day's work at least.
My main question: **Does anyone have a direct link to a detailed changelog or breaking changes log for the Sentinel ingestion APIs?** The general Azure Monitor changelog is too broad, and I'm sifting through a mountain of GitHub issues. I'm also curious if others who built "DIY" connectors have hit this recently and what your workaround was.
More broadly, this is a pain point – I keep a detailed comparison spreadsheet of log ingestion methods (custom API, DCRs, Logic Apps, the AMA agent), and the stability of the custom API path was a big plus for complex, pre-processed data. If this is a permanent shift, I'll need to re-evaluate.
Any pointers or shared experiences would be massively appreciated!
Test everything, trust nothing
Ugh, classic quiet Azure API pivot. Check the `api-version` param first. That old `2016-04-01` version on the DCR endpoint might have finally been sunsetted. Try `2023-01-01`.
Also, the auth error could be a red herring. They moved to Data Collection Rules (DCR) for custom log ingestion a while back. Your old direct-to-workspace endpoint might now require a DCR-based intermediate step and a different authentication flow.
I had a similar scare last quarter. My script was fine, but the service principal needed the new `Microsoft.Insights/dataCollectionRules/write` role assigned on the DCR, not just the workspace. The docs are, as usual, a maze of redirects.
Quick test: can you post to the DCR endpoint instead? ` https://{region}.ingest.monitor.azure.com/dataCollectionRules/{dcr-resource-id}/streams/{stream-name}`
- elle