Just spent an afternoon getting Panther alerts to properly trigger PagerDuty incidents with the right priority levels. It's not *hard*, but there's a couple of gotchas in the setup, especially around mapping Panther's severity levels.
The key is in the Panther alert forwarding rule. You'll define a `pagerdutySeverity` in the rule's `outputs` based on Panther's `severity` field. Here's a snippet from my rule logic:
`severity_mapping = {
"CRITICAL": "critical",
"HIGH": "error",
"MEDIUM": "warning",
"LOW": "info"
}
pagerdutySeverity = severity_mapping.get(severity, "info")`
Then, in PagerDuty, you create corresponding urgency rules based on that incoming severity. Don't forget to enable the "Send as PagerDuty V2 Events" option in Panther's PagerDuty destination config. Makes all the difference for the payload structure.
Biggest win? Having my `CRITICAL` data quality alerts wake people up, while `LOW` insights just log a ticket. Game changer for response times. Let me know if you run into mapping issues.
—b
—b
Thanks for sharing that mapping snippet, it makes sense. I'm just starting to set up something similar. When you say "create corresponding urgency rules in PagerDuty," does that mean you're using event rules on the PagerDuty service to translate the `critical` or `warning` severity into an incident priority? Or is the priority set directly in the payload from Panther?
Also, was there a reason you mapped HIGH to "error" instead of "critical"? I'm assuming it's to differentiate the top two levels in your on-call rotations.
Yeah, that V2 events checkbox is the sneaky part. I missed it the first time and nothing mapped correctly. Also, watch out for the `dedup_key` field in your rule. If you're not careful, it can cause duplicate incidents in PagerDuty if your logic doesn't create a truly unique string. I've started using a combo of the alert ID and a timestamp hash to keep it clean.
I like your severity mapping, it's similar to ours. One thing we did differently was to use the `custom_details` field in the PagerDuty payload to pipe in the original Panther severity and rule name. That way, even if it shows as 'error' in PD, the on-call engineer still sees 'HIGH' immediately in the incident details. Saves a click into Panther.
The `dedup_key` issue is real. A poorly constructed one can flood your on-call with noise. I've found using a hash of the core alert payload fields (like rule_id, resource, and generated_at) works more reliably than timestamp-based combos, as it prevents duplicates for the same logical event across any retries.
Including the original Panther data in `custom_details` is smart for context, but I'd also add the Panther investigation link there. Engineers shouldn't have to hunt for it.
On V2 events, missing that checkbox essentially breaks the severity mapping because the field names differ between V1 and V2 payloads. It's a silent failure mode that's easy to overlook during testing.
sub-100ms or bust