Integrating Qualys vulnerability data into Microsoft Sentinel presents a classic data ingestion and normalization challenge that many enterprise security architectures struggle with. The core issue is not merely the ingestion of raw XML or CSV files, but the transformation of that data into meaningful, actionable security incidents within the KQL schema, while maintaining context with your CMDB and other security signals. The "best way" is a multi-layered approach focusing on reliable ingestion, schema alignment, and operational workflows.
Based on several multi-cloud implementations, I recommend a pipeline architecture rather than a direct connector. The built-in Data Connector for Qualys VMDR is functional for basic use but often lacks the granular control required for sophisticated correlation.
**Proposed Architecture:**
* **Ingestion Layer:** Deploy the Qualys API-based Azure Logic App (or a containerized Python agent in AKS if you require higher throughput) to periodically pull vulnerability data. This provides superior logging, error handling, and scheduling control compared to the native connector.
* **Transformation & Enrichment Layer:** This is critical. Use an Azure Function (or Logic App actions) to parse the Qualys payload and map it to the `SecurityAlert` or a custom `Vulnerability_CL` table in your Log Analytics workspace. Enrich the findings with asset ownership data from your Azure Resource Graph or ServiceNow CMDB API.
* **Orchestration & Analytics Layer:** Within Sentinel, develop analytic rules that operate on this normalized data. Correlate vulnerabilities with active threats (e.g., a vulnerable software version paired with exploit activity from the `SecurityEvent` or `CommonSecurityLog` tables).
**Key Technical Considerations:**
* **Schema Design:** Do not dump raw Qualys XML into a custom table. Structure it. A sample normalized schema for a custom log table should include:
```kusto
Vulnerability_CL | extend AssetId = tostring(AssetId_s),
CveId = tostring(CveId_s),
Severity = toint(Severity_d),
Qid = tostring(Qid_s),
LastDetected = datetime(LastDetected_t),
Treatment = tostring(Treatment_s) // e.g., "Open", "In Progress", "Remediated"
```
* **State Management:** Qualys data is stateful. Your pipeline must handle the update of existing vulnerability records (e.g., status change from "Open" to "Fixed") to avoid stale alerts. This often requires a lookup step in your ETL logic.
* **Cost Awareness:** Ingesting every vulnerability finding from a large estate can be cost-prohibitive. Implement filtering at the ingestion layer—perhaps by severity, asset criticality, or only newly discovered findings—to control Log Analytics costs.
The ultimate goal is to move beyond a simple vulnerability dashboard. The best practice is to create fusion alerts where Sentinel correlates a high-confidence vulnerability with a corresponding exploit detection or anomalous access pattern, thereby elevating it to a genuine, prioritized security incident. Without this enrichment and correlation, you are simply creating a ticket generator for your SOC team, which adds little to your actual security posture.
Boring is beautiful
I'm a junior security engineer at a mid-sized financial services company (around 1k employees). We run a hybrid Azure setup and I manage the Sentinel ingestion pipelines for our vulnerability data in production, including Qualys.
I compared the built-in connector to a custom pipeline:
1. **Total monthly cost:** The built-in connector costs you nothing extra beyond your Sentinel ingestion fees. A custom Azure Logic Apps pipeline ran us about $35-50/month, and that's before the compute costs for any transformation scripts.
2. **Initial setup time:** The Sentinel Data Connector took an afternoon to configure and authenticate. Building a reliable Logic App with the Qualys API, error handling, and logging took me nearly a full week to get right.
3. **Data control and normalization:** The native connector drops data into the `CommonSecurityLog` table with a fixed schema. Our custom pipeline writes to a custom log table (`Qualys_Vulnerabilities_CL`), letting us reshape fields and add tags before ingestion. This was mandatory for us to join with asset owner data from ServiceNow.
4. **Maintenance and failure rate:** The Microsoft-managed connector has had zero downtime for us in 8 months. My Logic App fails about once a month due to Qualys API timeouts or schema changes, requiring manual review and restart.
I'd use the built-in connector unless you absolutely need to transform the data before it lands in Sentinel. If you need to pre-join data or change the schema fundamentally, then build a pipeline. To decide, tell us: do you need to enrich the vulnerability data with external owner/priority data *before* it hits your logs, and what's your team's tolerance for maintaining custom code?
Containers are magic, but I want to know how the magic works.
You're spot on about the need for a pipeline over the straight connector for real control. That transformation layer you mentioned is the make-or-break point in my experience.
One thing I'd add: don't sleep on using Azure Functions with a timer trigger for the ingestion instead of Logic Apps if you're already a code-first team. It's easier to version control, test, and you can bundle your transformation logic right in there before shooting it to a Log Analytics custom table. I've seen teams get burned in Logic Apps when their XML parsing needs got too complex for the built-in actions.
Also, enriching with CMDB data *before* it hits Sentinel is key, otherwise your SOC is stuck doing manual lookups. We pull asset owner and environment tags from ServiceNow at ingestion time, which makes the KQL for creating high-priority incidents way cleaner.
security by default
Absolutely agree about the CMDB enrichment being a game-changer. Pulling that data in early saves so much time later.
Your Azure Functions point is a good one for dev-centric teams. For those less code-comfortable, I've also seen Power Automate used as a middle ground - it can handle the API calls and basic transforms with a UI, and then pass the formatted data to a Logic App or directly to the custom table. Not as powerful as a full function, but it lowers the barrier.
One caveat on the pre-Sentinel enrichment: you have to build a solid caching or lookup strategy for that CMDB API call to avoid hitting request limits or adding too much latency to the pipeline.
Clean data, happy life.