Agree on the scheduled approach for audit findings - nightly control beats webhook reliability for that use case. The externalId hash is clever, though I'd use something like `hashlib.sha256(f"{evalGuid}:{resourceId}").hexdigest()` for the composite key.
One caveat on storing the last poll timestamp: I'd avoid S3 for that. It adds eventual consistency concerns. Parameter Store is better, but for a Step Function, you're better passing the timestamp as part of the execution input/output and letting the workflow state manage it. That way the schedule and state are coupled, which is simpler.
The real trick is the Jira query by externalId during a bulk run. If you're processing hundreds of findings, that's hundreds of GET requests. We built a small cache in memory for the run duration - query all existing tickets with the relevant externalId pattern once, then check locally. It cuts the API calls dramatically.
Cloud cost nerd. No, I don't use Reserved Instances.
I've been trying to get this working too and hit the same wall with the built-in webhooks. The mapping just doesn't fit JSM.
I ended up writing a small AWS Lambda to handle it. For the deduping, I'm using the Lacework alert ID as a custom field in Jira, then checking for that before creating. It's not perfect but cuts down repeats.
Quick question about auth - you mentioned service accounts. Are you storing the Jira API token in Lambda environment variables or something like Secrets Manager? I'm worried about that being exposed in logs.
Secrets Manager is the better option for the token. Lambda environment variables are encrypted at rest, but they're visible in plaintext if you have console access or in function configuration exports. Secrets Manager adds an extra access control layer and automatic rotation helps too.
Your deduping method sounds similar to what we did. The only issue we ran into was older findings that fired before we added the custom field - they'd create duplicates. We added a one-time backfill script to tag those.