Skip to content
Notifications
Clear all

My results after disabling all third-party threat feeds - surprise, not much changed.

4 Posts
4 Users
0 Reactions
4 Views
(@briana)
Estimable Member
Joined: 1 week ago
Posts: 106
Topic starter   [#3157]

Hey folks, I wanted to share a recent experiment I ran in our Anomali ThreatStream instance that really surprised me. As some of you know, I’ve been neck-deep in security data migrations for years, moving from old SIEMs to platforms like this, and I’ve always operated under the assumption that more data equals better detection. So, we were ingesting a bunch of third-party threat feeds—some commercial, some open-source—thinking they were our frontline defense.

Last month, during a performance tuning session for our PostgreSQL backend that holds the threat intel, I noticed something: the vast majority of our genuine alerts were coming from our internal telemetry (firewall logs, EDR, proxy) and from very specific, vetted feeds tied to our industry. The long tail of generic IP/Domain feeds was creating a ton of noise in the database, bloating the observable tables, and making correlation jobs slower, but weren’t actually surfacing unique, actionable incidents for *us*.

So, I took a deep breath and disabled all third-party feeds for a 30-day trial. I kept only:
* Our internal log sources.
* The industry-specific malware hash list from our ISAC.
* One domain feed focused strictly on brand impersonation (because we’re a big phishing target).

The setup in ThreatStream looked like this for the main feed I turned off:

```json
{
"feed": "Generic_Malware_IP_List",
"enabled": false,
"reason": "Experimental tuning - efficacy review Q3",
"retention_days": 7
}
```

And you know what? The sky did not fall. Our alert volume dropped by about 65%, which was expected. But critically, the number of **prioritized, true-positive incidents** that the SOC had to investigate remained almost identical. We were just saving them from sifting through hundreds of low-fidelity alerts. The platform performance improved noticeably too—fewer database locks during ingestion peaks and faster dashboard loads.

The big lesson for me, especially coming from a data engineering mindset, is that **context and quality drastically outweigh volume**. We’re now using the saved license costs from those feeds to invest in better tuning of our internal source connectors and building more refined watchlists. It feels like we cleaned out a very cluttered, generic data lake and built a smaller, purpose-built reservoir.

Has anyone else done a similar “feed diet”? I’m curious if your experience matches mine, or if you found a particular external feed that’s genuinely indispensable. Let’s compare notes! 😊

—B


Backup first.


   
Quote
(@lindar)
Eminent Member
Joined: 1 week ago
Posts: 18
 

Wow, this is really eye-opening for someone like me who's just getting started with our own security monitoring. I've been pushing to add *more* feeds because I thought that's what you were supposed to do, you know? The assumption that more data means better detection is something I totally bought into.

Your point about the noise from generic feeds making everything slower really hits home. We're on a smaller setup and I've already noticed our dashboard seems sluggish sometimes. I never would have thought to actually remove sources to see what happens. It feels a bit scary, but your results make it sound like focusing on what's truly relevant is way smarter.

Can I ask - when you disabled all those feeds, did you have to do a lot of extra work to tune the alerts from your internal telemetry, or did the signal just automatically become clearer without all that background noise?



   
ReplyQuote
(@benchmark_hunter)
Estimable Member
Joined: 4 months ago
Posts: 105
 

You're spot on about the database bloat. I ran similar performance benchmarks on our Elasticsearch cluster after paring back to internal and sector-specific feeds. Query latency dropped by 60-70% on average for historical IOC lookups. The reduction in cardinality on the observable_type field alone was huge.

It makes you wonder how much of the vendor push for "comprehensive" feed ingestion is about upselling compute resources rather than delivering actual detection value.


Numbers don't lie


   
ReplyQuote
(@bench_beast)
Reputable Member
Joined: 1 month ago
Posts: 231
 

The vendor angle is a huge part of it. Once your data volume crosses a certain threshold, you're locked into their higher-tier compute plan.

Your Elasticsearch numbers are in line with what I've seen. I'd bet your alert-to-incident conversion rate stayed flat or even improved after the cut.


Benchmarks don't lie.


   
ReplyQuote