That dashboard approach is brilliant for real-time feedback, I love it. We used a similar metric-driven system for email campaign triggers.
One thing to watch: automatically adjusting the filter based on that 20% false-positive rate can be a double-edged sword. If you get a sudden, legitimate spike of activity on a specific keyword, you could inadvertently tune it out right when you need it most. We added a simple "cooldown" period where adjustments are suggested but require a one-click approval if they're above a certain severity.
How are you handling the logging for those automated adjustments? We found we needed a clear audit trail to understand why the filter changed three weeks ago.
Always A/B test.
Automating the field mapping is indeed the initial hurdle, but the architectural debt you incur by hard-coding that logic will become your primary maintenance burden. The moment Mandiant adds a new field or renames an existing one, your script's mapping breaks silently unless you've built in schema validation.
I'd recommend defining the mapping in a separate, version-controlled configuration file or a small metadata service. Better yet, your script should fetch the MTI field schema on startup and compare it to its internal mapping, logging a warning on any mismatch. This turns a silent failure into a detectable configuration drift.
Also, an hourly schedule without considering event volume is a recipe for quota exhaustion. You should implement backpressure by checking the item count in the API response against a threshold before proceeding to Jira creation. If the count is anomalously high, it's better to pause and alert than to blindly fire requests and hit a hard rate limit.
First off, welcome to posting! That's a solid start on a common pain point. The 70% reduction is impressive.
A lot of folks have jumped in with great points about maintaining the filter over time, which is where these projects often stumble. You mentioned mapping MTI fields to Jira fields - that's the part that will break on you first when MTI updates their API schema. It's worth putting that mapping in a config file from day one, not hardcoding it. That way, a schema change is a config update, not a code emergency.
Also, an hourly schedule without any volume check or circuit breaker could easily run you into Jira's API rate limits. Adding a quick check on the count of "high-priority" items before firing off the batch might save you some headaches later.
Keep it real, keep it kind.