We've got Lacework alerting to Slack and PagerDuty, but our change process requires a Jira Service Management ticket. Need to automate creating a JSM ticket for every High/Critical Lacework finding.
Tried their webhooks, but the JSON payload doesn't map cleanly to JSM fields. Also need it to only fire for new findings, not every re-evaluation.
Current workaround is a clunky Python script that:
1. Polls Lacework APIs
2. Filters for new High+ alerts
3. Uses Jira REST API to make tickets
It's fragile. Want something more direct.
Has anyone built a reliable integration? Key requirements:
* Must populate JSM custom fields (priority, summary, description).
* Should avoid duplicate tickets for the same finding.
* Needs to attach relevant evidence (like the Lacework alert ID).
Looking for:
* Example webhook transformation logic (Node or Python).
* How you handled authentication (API keys, service accounts).
* Any pre-built connectors or tools you used.
// chris
metrics not myths
Been there with the clunky script approach, Chris. The webhook mapping is a pain because Lacework's payload is so nested. We ended up using a small AWS Lambda (Python) as a transformer.
It listens for the Lacework webhook, flattens the key bits like alert ID and severity into a format JSM expects, and adds a hash of the finding as a custom field to prevent dupes. For auth, we store the Jira service account API key in AWS Secrets Manager. The lambda fetches it on execution.
One gotcha: Lacework will send the same finding on re-evaluation. We stamp a "last_processed_time" in a DynamoDB table against the alert hash and only create a ticket if it's truly new (like, over 24 hours old). Stops the ticket spam. I can share the gist of the flattening logic if you're on Python.
it worked on my machine
Lambda as a transformer is solid. For de-duping, a hash is smart, but we've also added a check for Jira's "external ID" custom field. If we see the same Lacework alert ID there, we just add a comment to the existing ticket instead of creating a new one. Saves the DynamoDB table.
On auth, we went with a Jira API token in an Azure Key Vault since our shop is Azure-heavy. The lambda grabs it at runtime. If you're in AWS, Secrets Manager is definitely the way to go.
The flattening logic is the tedious part. Main trick? Use `json_normalize` from pandas in your Python lambda - it saves a ton of nested looping to get at the details like resource ID and recommendation.
Automate everything.
DynamoDB for deduping works but adds cost and latency. We skip it entirely.
We hash the finding's composite key (alertId + resourceId + evalGuid) and set it as the Jira ticket's `externalId`. On any incoming webhook, we first query Jira by that custom field.
- If ticket exists, we append a new comment with the updated timestamp.
- If not, we create.
No external state needed. Query is cheap, and Jira becomes the source of truth for ticket state. One lambda, no DDB table.
Metrics don't lie.
Great approach! Using Jira itself as the dedupe source is clever and cuts out an extra service. We did something similar but added a fail-safe for the query step.
If the Jira query times out or is slow, the lambda could get throttled. We added a simple in-memory cache of recently processed hashes (last 10 minutes) as a fallback. It's not perfect, but it prevents a brief Jira API hiccup from accidentally spawning duplicate tickets during a spike in alerts.
That's a neat detail on using `alertId + resourceId + evalGuid`. We found that `alertId + resourceId` was enough for our vuln findings, but I can see why you'd include the evalGuid for compliance events.
Data doesn't lie, but dashboards sometimes do.
Lambda as middleware just moves the fragility. You're still maintaining a custom transformer, plus now it's a black box running on someone else's dime.
What if Lacework changes their webhook schema again? Your lambda breaks. What if Jira's API changes? Your lambda breaks. You've swapped a script for a serverless function with the same core problem.
Polling the Lacework API might actually be more robust. You control the frequency and handle errors explicitly. A direct webhook expects the world to be perfect. It rarely is.
Doubt everything
The Python script isn't inherently fragile - the fragility is in your orchestration and error handling. You're already 90% there. The core logic should stay a script, but run it as a scheduled Fargate task or a Step Function with a Lambda. That gives you logging, retries, and version control.
Key difference from the webhook approach: you control the schedule. A nightly run for audit findings is more reliable than real-time webhooks that can get dropped. Store your last successful poll timestamp in a simple S3 object or a Parameter Store value.
For deduping, hash the finding composite key and make that the Jira ticket's `externalId`. Your script queries Jira by that field before creation. No need for a separate database.
Treat your script like production code. Parameterize secrets, add structured logging, and handle Jira API rate limits. That's more direct and maintainable than adding a middleware transformer.
Where is your SOC 2?
That `json_normalize` tip is a lifesaver for flattening those nested alerts. Saves so much messy parsing code.
I'd just add a caution on pandas in Lambda, though. The package size can bump you into slower cold starts. For simpler payloads, Python's built-in `jsonpath-ng` library can sometimes do the trick with a smaller footprint, especially if you're only plucking a few specific fields.
Great call on using Jira's external ID for deduping. We found that adding a comment to the existing ticket, like you mentioned, helps keep the audit trail clean without creating noise.
`json_normalize` works but the pandas layer adds 60MB. If you're just pulling fields like resourceId and recommendation, Python's built-in `json.loads()` with a simple dict lookup is lighter. Lambda cold starts matter when you're processing webhooks in near real-time.
Querying by external ID for deduping is the right move. We do the same. The comment addition is good for audit trails, but watch for Jira's API rate limits if you get a flood of re-evaluated findings for the same ticket. We hit that once.
Benchmarks don't lie.
Totally with you on the pandas overhead for a simple transformer. That cold start tax is real when you're trying to keep this pipeline responsive. I've resorted to a small dictionary mapping that defines the JSON paths for the fields I need, then uses a loop to extract them - keeps the package lean.
And the rate limit point is crucial, especially for compliance scans. We learned that the hard way when a cloud resource was re-evaluated every hour, flooding one ticket with comments. We now batch comment updates into a single API call if possible, or at least implement an exponential backoff on the lambda retry.
Implementation is 80% process, 20% tool.
That dictionary mapping approach for field extraction is a smart optimization, especially if the Lacework payload structure is relatively stable. We use something similar with a list of prioritized field paths; if the primary path is missing, it tries a fallback.
Your point on batching comments is critical for operational sanity. We also found that Jira's rate limiting gets triggered faster than the documented thresholds suggest, particularly in multi-tenant instances. Our current logic suppresses comment updates if the finding state hasn't materially changed - for example, if a compliance check flips from "failed" to "evaluating" and back to "failed" within a short window, we don't add another comment. It cuts down on the noise and the API calls.
Support is a product, not a department.
The external ID as a dedupe key is the obvious approach. It's surprising more people don't default to it.
Your composite key is solid for most cases, but watch for Lacework's system-generated alerts where the `evalGuid` might be null. That'll break your hash. We filter those out or fall back to a timestamp.
Also, the "query is cheap" assumption can bite you during a major incident. If a thousand findings fire at once, that's a thousand immediate Jira GET requests before any tickets are even created. You'll need aggressive exponential backoff in your retry logic, or you'll just trade DDB latency for Jira API throttling.
Your fancy demo doesn't scale.
The built-in Lacework to Jira integration in their UI actually maps fields directly, but only for Jira Software projects. It doesn't work for JSM yet. For your webhook mapping, we used a small dictionary to define the JSON paths from the Lacework alert to the JSM field IDs, avoiding any heavy libraries. Authentication was the trickiest part - we used a dedicated Jira service account with an API token, stored securely as an environment variable in our middleware, never in the script itself.
For duplication, querying Jira by the Lacework alert ID in the description field worked as a simple gate before ticket creation. It's not perfect, but it's resilient to API changes. Have you considered using Lacework's alert rules to only send webhooks for net-new findings? That could simplify your filtering logic considerably.
Stay curious, stay critical.
You're right that the fragility doesn't magically disappear with Lambda. But you're trading one set of headaches for another.
>Polling the Lacework API might actually be more robust.
Maybe, but now you own the scheduling, state management, and idempotency. You also have to handle rate limits on *both* ends and store your last poll marker somewhere durable. That's more moving parts, not fewer.
A webhook failing is obvious - it's an event that didn't arrive. A scheduled poll that silently breaks because your timestamp store got corrupted? That can go unnoticed until your next audit. At least a broken webhook pipeline usually screams.
You're right about the polling headaches. The core issue with the webhook approach is the field mapping, which you can solve without a full polling architecture.
For the transformation, avoid heavy libraries. I use a simple dictionary mapping the Lacework JSON paths to Jira field IDs. Something like:
field_map = {
'summary': 'data[0].eventName',
'customfield_12345': 'data[0].recommendationId'
}
Then iterate through the map, using `.get()` on the parsed JSON object. For auth, a dedicated Jira service account with an API token stored in Secrets Manager works. The token goes in the `Authorization: Bearer` header for all REST calls.
The bigger catch is the re-evaluation noise. You can filter at the source by adjusting your Lacework alert rule to trigger only on status changes, not every re-evaluation. That cuts the webhook volume significantly before your code even runs.
Every dollar counts.