Looking for the most direct method to feed external threat intelligence feeds (IPs, domains, URLs) into Splunk ES. I manage vendor-sourced feeds and need to operationalize them for correlation and alerting.
Specifically, how do you handle the ingestion and normalization? Do you use the Threat Intelligence Management framework, a custom lookup, or a different method? I'm concerned about performance and maintenance overhead. Concrete examples of your process would help.
I'm a junior security analyst at a 500-person SaaS company where we run Splunk ES with around five different threat intel feeds for our SOC. We've been using the Threat Intelligence Management (TIM) framework for about a year.
Core comparison of methods:
Setup and Maintenance: TIM adds a specialized lookup table (threat intel) you have to manage, which took us a day to configure properly. A custom CSV lookup is simpler at first but becomes manual overhead.
Data Format Handling: TIM requires a specific stanza format (like `key, threat_type, source`) in your source files. If your vendor feed doesn't match, you need a preprocessing script. We use a small Python script for one feed that adds the required fields.
Performance and Scale: TIM is designed for large volumes; our feeds total about 500k indicators and we haven't seen a performance hit on searches. With a basic lookup, you risk search slowdowns once you pass a few hundred thousand rows unless you tune it.
Integration and Alerting: TIM wins here. It's built for ES, so your threat intel is automatically available for correlation searches and risk-based alerting. Using a regular lookup means you have to manually join data in every search.
My pick is the TIM framework for any ES environment that needs more than one feed or wants to use the intel for automated correlation. If you're only testing a single, simple feed, a lookup might be okay to start. To decide, tell us roughly how many indicators you're managing and if your feeds are already formatted for Splunk.