Exactly, and that ASN/SSL tagging approach is a solid move towards transparency. The next hidden variable we found was in the CDN layer. A high percentage of our false positives from RF's risk score were domains fronted by Cloudflare or similar services, essentially inheriting a clean infrastructure reputation they hadn't earned.
We started pulling the `CF-IPCountry` header and the upstream IP from our own perimeter logs. If a "high risk" domain was fully proxied through a major CDN, we could weight it lower, because the actionable infrastructure signal was obscured. It's another whack-a-mole layer, but it further isolates the vendor's specific heuristic from our own threat model.
—Alex
The tuning treadmill you describe is the hidden operational cost of these feeds. We tried the same thing with a DNSBL as a third source and found the same result, just three timestamps to argue over instead of two.
Your moving target problem gets worse when a vendor silently changes their scoring weights. We once had a whole week of alerts because RF started weighting domain age more heavily, and our old thresholds were calibrated for their old registrant-focused model. There was no changelog.
Logs don't lie.
That low correlation figure on new domains lines up with what I've seen when testing these feeds for health score integrations. The methodology gap you've identified is critical.
You mentioned the lack of transparency around source quality weighting. I've found that's where the biggest discrepancies hide. For example, a VT score might be heavily influenced by a single, less-reputable scanner flagging something as 'suspicious', while RF could be ignoring that scanner's data entirely in favor of proprietary link analysis. Your pipeline would need to account for that variance in evidence credibility, not just the final score.
Your point about timeliness is another key factor. In our tests, RF often scored domains as high-risk based on predictive indicators like registrant patterns *before* any malicious payload was deployed, which VT wouldn't see. This creates a lagging vs. leading indicator problem for automation. Have you considered adding a simple 'days since first detection' field to your context log to track that latency? It helped us qualify disagreements.