I've been benchmarking vulnerability detection workflows across several platforms (Wiz, Tenable, Qualys) that ingest CrowdStrike's intelligence feed, and a key input metric is the accuracy of the intel itself. False positives waste engineering cycles, while missed critical vulnerabilities pose direct risk.
My team tracks a simple accuracy score derived from our internal validation pipeline. We classify intel as "validated" if a patch is applied and the threat is mitigated, "false positive" if the patch introduces no observable security improvement or the CVE is contested, and "missed" if a critical CVE appears in other sources but not in our primary feed. The formula we use in our dbt models is:
```sql
with validation_summary as (
select
date_trunc('month', ingested_at) as intel_month,
count(case when status = 'validated' then 1 end) as validated_count,
count(case when status = 'false_positive' then 1 end) as false_positive_count,
count(*) as total_reported
from prod.vuln_intel_validation
where ingested_at >= '2024-01-01'
group by 1
)
select
intel_month,
validated_count,
false_positive_count,
total_reported,
round((validated_count::decimal / total_reported) * 100, 2) as precision_pct,
round((false_positive_count::decimal / total_reported) * 100, 2) as false_positive_rate
from validation_summary
order by intel_month;
```
Our preliminary 2024 data shows a slight improvement in precision (approx. 3% increase year-to-date) compared to the same period last year. However, this could be noise.
I'm looking for community data points to see if this aligns with broader experience. Specifically:
* Has there been a noticeable reduction in "noise" CVEs (low severity, disputed, or non-exploitable) in the feed?
* For critical vulnerabilities (CVSS >= 9.0), is the time-to-detection latency improving, and are the associated threat actor profiles more accurate?
* Are you adjusting internal risk scores less frequently after the intel is ingested, indicating higher initial accuracy?
Concrete numbers or observable changes in your workflow would be most valuable for comparison.