Step-by-step: Integrating runtime defense alerts into our PagerDuty escalation chain.

We've been running Palo Alto Prisma Cloud for container and host vulnerability scanning for a while, but their runtime defense alerts were just piling up in a Slack channel where everyone ignored them. That's useless noise. The goal was to get critical runtime events—like a container spawning a reverse shell or writing to a sensitive directory—into our on-call rotation via PagerDuty with proper severity mapping.

Here's the workflow we built and the config that makes it tick. The key is filtering the signal from the noise; you don't want every low-severity alert waking someone up.

**Step 1: Prisma Cloud Webhook Configuration**
You configure a webhook in Prisma Cloud to point to a small middleware service (we used a serverless function). Prisma's native PagerDuty integration is too rigid for our severity mapping needs.

**Step 2: Filtering & Enrichment Logic**
Our middleware parses the Prisma Cloud alert JSON, applies our business logic, and forwards only designated alerts to PagerDuty. Critical alerts go to a high-urgency PagerDuty route, others to a low-urgency one.

Example alert filter logic (Node.js snippet):
```javascript
// Only forward these specific runtime event types
const CRITICAL_EVENT_TYPES = [
'Runtime Container Exec',
'Runtime Container Reverse Shell',
'Runtime File System Modified'
];

function shouldEscalate(prismaAlert) {
// Check if it's a runtime defense alert and of high severity
if (!prismaAlert.type?.includes('Runtime')) return false;
// Apply custom filters, e.g., ignore certain namespaces
if (prismaAlert.resource?.namespace === 'monitoring') return false;
return CRITICAL_EVENT_TYPES.includes(prismaAlert.detailedType);
}
```

**Step 3: PagerDuty Payload Transformation**
Prisma's alert schema isn't 1:1 with PagerDuty's V2 event format. You need to map fields.

Key mapping in the middleware:
- `event_action: trigger` for new alerts, `resolve` for clear events.
- Map Prisma severity to PagerDuty severity (we map "high" to `critical`, "medium" to `warning`).
- Include the Prisma alert ID in `dedup_key` for proper incident resolution.

**Step 4: Infrastructure as Code**
The whole pipeline is defined as Terraform for the serverless function and PagerDuty services, and a CI/CD pipeline deploys it. No manual configuration.

**Pitfalls we hit:**
- Prisma's webhook payload can be large; ensure your endpoint can handle it.
- Alert "clear" events don't always match the "trigger" event structure. Your logic must handle both.
- Without careful filtering, you'll get alert storms. Start with a very restrictive allow-list of event types.

The result is a reliable pipeline where a genuine runtime threat creates a PagerDuty incident within 15 seconds, and the on-call engineer has all the Prisma Cloud investigation links right there. It finally makes the runtime data actionable.