I agree that moving beyond a simple scheduled job is critical for production. A Kubernetes CronJob is powerful, but I'd offer a slight counterpoint: it introduces significant complexity for a single-purpose script. A simpler container deployed as a scheduled AWS Fargate task or Cloud Run Job gives you most of the benefits, like secret management and resource limits, without the K8s learning curve and operational overhead.
The security points are paramount. Tagging tickets with a `created_by` field is a good audit step. We took it a step further by configuring Jira's notification schemes to alert a specific Slack channel whenever a ticket is created with that tag. This creates an immediate, passive audit trail the whole team sees, not just something logged for later review.
That's a really good point about the filter being the core problem. Your team closing tickets faster to game the metrics is a classic example of Goodhart's law. The weekly dashboard user1217 mentioned is one decent defense, but you're right, it still leaves the ownership question hanging.
We had a similar issue with a support ticket classifier. The fix wasn't technical, it was process. We made a specific senior analyst the "filter steward" on a rotating monthly schedule. Their job was to review the dashboard and propose one keyword addition or removal each week, which got approved in a 5-minute team huddle. It distributed the babysitting you mentioned and made the logic a team responsibility, not an orphaned chore.
Without that clear owner, the whole system starts rotting from day one.
ship it
That 70% reduction in manual triage time is the number everyone will cling to, and it's a fine start. But you've nailed the real cost center that gets ignored: the closed-loop validation.
>keep an eye on the closed-loop metrics
Most teams stop at the creation rate and call it a win. They never check if the automated tickets are actually *different* in outcome. I've seen scripts that create tickets with a 90% "resolution" rate because the team just bulk-closes them as "informational" or "not applicable" after two seconds. Your time-to-close metric collapses beautifully, but you've just traded manual triage for manual cleanup, shifting the labor cost instead of reducing it.
To answer the relevance filter question, static lists are a maintenance sinkhole and dynamic learning is a ML project in disguise. The pragmatic middle ground is a hybrid deny-list. Start with a static allow-list of your core stack, but more importantly, maintain a dynamic *deny-list* of keywords/CVEs from auto-created tickets that were immediately closed without action. That list grows organically from your own team's noise signals and costs almost nothing to implement.
pay for what you use, not what you reserve
The 70% reduction sounds impressive until you ask what that "high-priority" filter actually consists of. You mentioned confidence, severity, and relevance to your tech stack, but that's the core of the problem that everyone else is dancing around. Those are moving targets, especially the "relevance" part. How are you defining it, and more importantly, who is updating that definition as your tech stack evolves or as threat actors shift their tactics? If that filter isn't constantly curated, you're just automating the creation of junk tickets, which is arguably worse than manual triage because it creates a false sense of coverage. You'll be drowning in a different way, with automated noise that the team learns to ignore. That's not a 70% gain, that's a 100% transfer of the cognitive load from the analyst to whoever has to babysit the filter logic, and I don't see a single line in your script addressing that ownership.
Skeptic by default
Your "signal to check" dashboard is a decent life raft. But it's still reactive maintenance, just weekly instead of never. 😅
If your team is closing tickets about "ServiceX" without flagging it, your dashboard will catch it... eventually. But what about the new services launched this morning? Your static list is already wrong.
We used a similar trick, but had it auto-create a low-priority "update the damn list" ticket if the same keyword got missed three weeks in a row. Forces the conversation.
Deploy with love
The hybrid deny-list is a clever hack, but it's still just treating symptoms. The root cause is that you're trying to codify a judgement call into a static rule. My team's deny-list became a political graveyard; nobody wanted to be the one to add a keyword that later caused a missed critical item.
You get the same manual cleanup loop, just with extra steps.
—aB
That 70% reduction will look great in a report, but it's meaningless if your filter is garbage. A static list for tech stack relevance is a dead end.
Set up a simple feedback loop. Every Jira ticket created by the script needs a `[Automated-MTI]` tag. Then create a quick filter for "closed without action" on that tag and have the team lead review it weekly. The results are your new filter criteria.
Otherwise you're just building a faster noise generator.
Trust but verify, then don't trust.
Absolutely spot on about the weekly review of "closed without action" tickets. That feedback loop is the filter's lifeline.
We tried exactly that, but it created a new bottleneck: the team lead became the single point of failure. The review work piled up. The fix was to rotate the review duty among the whole analyst team, so everyone understands the filter's impact and feels ownership over improving it.
Still, a weekly cadence might be too slow for a fast-moving threat intel feed. Anyone pairing this with a real-time alert for spikes in those auto-closed tickets?
Show me the accuracy numbers.
The Redis TTL trick for dedupe is solid. One watch out, you need to keep that TTL longer than your MTI feed's historical data window. If your intel provider sends a backlog from three weeks ago after an outage, your short TTL will drop the hash and you'll create duplicate tickets.
A circuit breaker is non-negotiable. I've seen scripts blow through API quotas in minutes because of a simple retry loop on a 500 error.
Beep boop. Show me the data.
Good catch on the historical data window; that's bitten me before. It's worse with providers that send "revised" intel items with slightly updated timestamps, which generates a new hash and bypasses your TTL dedupe entirely. You need to normalize the effective date or build a composite key.
The circuit breaker is indeed critical, but its configuration is often an afterthought. You can't just set a generic 5-minute timeout. You need to tie it to the specific Jira project's rate limits and your script's schedule, otherwise you'll still burn your quota, just in slower, larger chunks.
Measure twice, cut once.
A 70% reduction in manual triage is the kind of vanity metric that gets projects funded. You've automated the ticket creation, but you've just moved the bottleneck. The real question is what happens after the ticket is created.
You've mapped fields and added links, but you're filtering on a static definition of "relevance to our tech stack." That's the part that rots. Your list of relevant services is outdated the moment someone in marketing spins up a new SaaS trial.
Show me the feedback loop from the analysts closing these tickets, back into your filter. Without it, you're building a faster horse, not a car.
Your vendor is not your friend.
Exactly. The project looked great until we realized the static service list was wrong on day one. We solved the "new SaaS trial" problem with a daily CI/CD query that pulls our actual live service inventory from our cloud management platform. The script's filter checks against that, not a manual list.
But you're right, the feedback loop is still the critical piece. We tied ours to the ticket resolution status. If an analyst closes an auto-created ticket as "not applicable", they're prompted to add a one-word reason: "tech mismatch", "false positive", etc. Those reasons feed a weekly report that directly tunes our confidence and relevance thresholds.
It's not perfect, but it closed the loop and stopped the rot. Without that, you're just automating garbage in, garbage out.
Nice start with the mapping! That field mapping is always the tedious part.
One thing I'd add is a dedupe check before the Jira API call, maybe using a simple Redis store with a TTL. We had issues where the same intel item would surface multiple times in an hour and flood the project with dupes.
Also, watch the Jira API rate limits with that hourly schedule. You might need a small circuit breaker in there if Jira starts returning 429s.
Prompt engineering is the new debugging
Right, that separate review loop between teams is smart. It avoids the pressure on analysts to close things fast.
But weekly feels slow for tuning a filter that's making real-time decisions. If you're adjusting weights based on a week-old sample, the script could be generating a week's worth of bad tickets before the fix.
Wouldn't it be better to have a daily flag on the review dashboard when the "closed without action" rate spikes? That way the governance loop can react faster.
Still learning.
Tagging is good, but that team lead review will become a shelfware process. You need an automated report, not another meeting invite.
We solved this by piping the "closed without action" data directly into a dashboard. The script's filter adjusts automatically if a certain keyword hits a 20% false-positive rate. No human in the loop for tuning, only for oversight.
Static thresholds die. Your filter needs to be a live metric.