Having recently completed a significant hardening project for our network perimeter, I wanted to document a scalable, data-driven methodology I implemented for dynamically blocking malicious IPs on our WatchGuard Firebox T-series appliances. The native "External Block List" feature is potent, but its manual curation is operationally fragile. My goal was to create a pipeline that automatically aggregates, deduplicates, and validates threat intelligence feeds into a Firebox-consumable list, with full lineage tracking.
The core architecture leverages a simple analytics engineering pattern:
* **Ingestion:** Multiple open-source and commercial threat feeds (e.g., abuse.ch, blocklist.de) are pulled via scheduled Python scripts.
* **Staging & Transformation:** Raw IPs are loaded into a staging table in our cloud data warehouse (BigQuery, but Snowflake/Redshift are analogous). A series of dbt models then:
* Deduplicates addresses across all sources.
* Applies basic sanity checks (valid IPv4 format, excluding private RFC 1918 ranges).
* Tags each IP with its source and first-seen timestamp.
* **Curated Output:** A final dbt model materializes a view of the validated, unique IPs from the last 24 hours. This view is then exported as a flat `.txt` file to an internal web server.
The critical piece is the Firebox configuration. Instead of a static list, it now points to the dynamically generated file via HTTPS. The relevant `proxycfg.cfg` snippet illustrates the setup:
```xml
DYNAMIC_THREAT_INTEL
https://internal-tools.domain.com/threat_feeds/blocklist_current.txt
ip
60
30
```
This list is then applied to the relevant policies. Key data quality considerations we monitor include:
* Daily count volatility (sudden drops may indicate feed failure).
* Overlap percentage between source lists to identify feed redundancy.
* Cardinality of the final list to prevent unintended performance impact.
The primary benefit is the transformation of a previously manual, stale control into a measurable, automated data product. We've observed a measurable decrease in probe traffic and a more consistent security posture. I'm particularly interested if others have implemented similar pipelines and how you handle false positive mitigation or tiered blocking strategies.
- dan
Garbage in, garbage out.