I've been evaluating Panther's alerting capabilities for our HR incident management workflows, and I need to route critical alerts to our on-call teams via PagerDuty with proper priority levels. After reviewing existing threads and documentation, I've compiled a step-by-step process, but I'm seeking validation on a few specific points.
My goal is to map Panther's alert severity to PagerDuty's incident urgency, ensuring that high-severity items like payroll processing failures create PagerDuty incidents with immediate escalation, while lower-severity notifications follow a standard workflow.
Here is my intended configuration process:
* **Step 1: Configure the PagerDuty Destination in Panther**
* In Panther, navigate to Destinations and create a new PagerDuty destination.
* Provide the Integration Key from the PagerDuty service you created.
* The critical setting here is configuring the `severity` mapping in the JSON payload template.
* **Step 2: Map Alert Severity to PagerDuty Payload**
* The default payload may not differentiate priorities. I modified the payload template to conditionally set the PagerDuty `severity` field based on Panther's alert context.
* For example, I used logic like:
`{% if alert.severity == 'Critical' %}critical{% elif alert.severity == 'High' %}error{% else %}warning{% endif %}`
* **Step 3: Create a Corresponding PagerDuty Service and Escalation Policy**
* In PagerDuty, ensure the service's escalation policy reflects the urgency levels sent from Panther. A `critical` severity should trigger an immediate phone call, while a `warning` might only create an email alert.
My primary questions are:
* Has anyone encountered issues with the payload customization where the severity mapping did not propagate correctly to PagerDuty's incident urgency?
* For HR-specific use cases (e.g., benefits system downtime vs. routine report generation), how have you structured your alert rules in Panther to assign the appropriate severity before it reaches PagerDuty?
* Is there a recommended method for testing this integration without triggering actual pages to the on-call team?
That's exactly the step where I got stuck last week. The default payload template felt too generic. I ended up using a custom template that checks Panther's `severity` field and sets the PagerDuty `severity` to `critical`, `error`, or `warning`. The key was getting the JSON syntax right inside the Panther UI for the conditional logic. Did you find a good example to copy from? I'm curious if you're mapping INFO level alerts too, or just skipping them.
Great call focusing on the payload template - that's where the real mapping happens. The default one doesn't give you the granular control you need for HR incidents.
For payroll failures, you'll want to catch anything `CRITICAL` or `HIGH` from Panther and set the PagerDuty severity to `critical`. That's what triggers the immediate escalation. Something like this in your template's custom JSON:
```json
{
"event_action": "trigger",
"payload": {
"summary": "{{ .Alert.Title }}",
"severity": "{{ if or (eq .Alert.Severity "CRITICAL") (eq .Alert.Severity "HIGH") }}critical{{ else if eq .Alert.Severity "MEDIUM" }}error{{ else }}warning{{ end }}",
"source": "Panther"
}
}
```
One caveat: PagerDuty's `urgency` field, which actually drives the on-call schedule, is separate from `severity`. You can set `urgency` to `high` in the same payload for those critical items to make sure they bypass any low-urgency filters on the service side. Did you run into any issues with the conditional logic syntax when you tested it? That `{{ if ... }}` block can be tricky to get right in their UI.
Integration Ian
That's a solid point about the `urgency` field. I found that even with `severity` set correctly, our PagerDuty service had a default urgency of 'low' that would override it. Adding `"urgency": "high"` to the payload for critical alerts was the fix.
Your template logic looks good for the mapping. One thing I'd add - for HR workflows, you might also want to include the specific detection rule ID or a link back to the Panther alert in the custom details. It helps our responders get context faster without jumping between tools. Did you embed any custom fields like that?
Hey, thanks for laying out your steps so clearly. I'm actually working on a similar setup for our finance team's alerts.
I got stuck at the same part in Step 2, mapping the severity field. The template logic for 'critical' and 'error' makes sense, but what are you doing with Panther's 'INFO' level alerts? Are you just not sending them to PagerDuty at all, or mapping them to a 'warning' to keep a record? I'm worried about creating alert fatigue.
Good question on INFO alerts. I filter them out entirely at the destination level - if it's not actionable for an on-call engineer, it shouldn't generate a PagerDuty incident. Creating a 'warning' just for a record adds noise and dilutes the urgency of real issues.
You can add a conditional rule in the Panther destination itself to only forward alerts where severity is MEDIUM or higher. That keeps the mapping logic clean and prevents alert fatigue from the start.
Totally agree with filtering at the destination level. It's cleaner and saves you from having to write logic for INFO in the template.
One thing I'd add: I also set up a separate, low-volume Slack channel for those INFO alerts. That way the team can still see them if they want, but they don't clutter the incident console. You can use a second Panther destination with a simple severity filter for that.
Also, good point about not diluting urgency. I've seen teams where warning-level PagerDuty incidents get auto-resolved by a webhook after 24 hours, but even that adds operational overhead.
Infrastructure as code is the only way
Good luck with that default payload. It's useless.
You're right about severity mapping being the critical setting, but you're skipping the actual step. What's your template logic? Vague goals don't build integrations.
And mapping to PagerDuty's severity isn't enough. Their "urgency" field controls the on-call schedule, and service-level defaults will stomp your settings. You need to set both. Miss that, and your payroll alerts won't escalate.
Just my two cents.