Skip to content
Notifications
Clear all

Step-by-step: Getting Snyk alerts into PagerDuty without alert fatigue.

9 Posts
9 Users
0 Reactions
0 Views
(@crm_hopper_2024)
Reputable Member
Joined: 4 months ago
Posts: 121
Topic starter   [#14772]

Alright, let's cut through the vendor fluff. Snyk's default alerting is a noisy mess. Routing everything to PagerDuty is a recipe for getting ignored.

Here's the sane path. First, filter at the source. Use Snyk's `severity` and `issueType` params in your webhook setup. Ignore low-severity libraries, only send high/critical for direct dependencies and *maybe* license issues. Then, in PagerDuty, use event rules to throttle or auto-resolve duplicate alerts. The goal is one actionable PagerDuty ticket per *real* problem, not 50 for the same vulnerable transitive dependency across repos. It's about signal, not volume.


CRM is a means, not an end.


   
Quote
(@hannahm)
Trusted Member
Joined: 1 week ago
Posts: 62
 

Okay, that sounds like a solid approach. Filtering at the source makes total sense to avoid the noise. When you say "maybe license issues," do you have a rule of thumb for when those actually warrant a page? We're just starting out and our legal team is... let's say less defined, so I'm curious what triggers a real action for people.

Also, how do you handle the auto-resolve in PagerDuty? Is it based on the exact same vulnerability ID across services, or do you group by something else? I'd be worried about silencing something that's actually a bit different.


Just my two cents.


   
ReplyQuote
(@jamesl)
Eminent Member
Joined: 5 days ago
Posts: 17
 

Filtering by `issueType` is crucial. I'd exclude "configuration" entirely for PagerDuty routing, as those are rarely page-worthy. The core challenge is transitive dependency noise. Even filtered to high/critical, you'll get duplicate alerts across services.

The PagerDuty event rule should deduplicate on a composite key: `vulnerability_id` + `root_project_name`. This groups the same flaw across different manifests in the same repo, preventing multiple incidents. Without the project context, you risk merging unrelated service issues.



   
ReplyQuote
(@jameson)
Trusted Member
Joined: 6 days ago
Posts: 44
 

Totally agree on filtering by `issueType`. We initially sent everything over and our on-call rotation revolted. The "configuration" alerts, like Dockerfile exposures, were constant noise but never an actual outage.

One extra filter that saved us: excluding development dependencies entirely in the webhook logic. Most high-severity vulns in `devDependencies` don't warrant a page at 3 AM.

How do you handle license alerts now? We ended up creating a separate low-priority PagerDuty service for those, so they still get tracked but don't wake anyone up.



   
ReplyQuote
(@baller_analytics)
Estimable Member
Joined: 1 month ago
Posts: 123
 

You're right about the composite key, but that's still treating symptoms. The root problem is alerting on transitive deps at all.

If it's transitive, it's not in your direct control. Paging someone at 3 AM for a library three levels deep in a framework you can't immediately patch is pointless. It creates tickets that just sit in "acknowledged" for weeks.

Filter further. Only route alerts for direct, non-dev dependencies. That cuts the noise by 80% before it even hits PagerDuty's deduplication logic.


If it's not a retention curve, I don't care.


   
ReplyQuote
(@carlr)
Estimable Member
Joined: 1 week ago
Posts: 92
 

Partially agree, but this assumes you only want alerts for things you can fix immediately.

If a critical transitive vulnerability is widespread and requires a coordinated framework upgrade, that *is* actionable. It shouldn't page at 3 AM, but it should create a tracked incident.

Your filter would miss, for example, a severe log4j-style flaw buried in a transitive dependency. The better approach is to use Snyk's `projects` or `tags` to only monitor production services, then let PagerDuty event rules suppress duplicates and set a low urgency.


Your fancy demo doesn't scale.


   
ReplyQuote
(@cost_optimizer_99)
Estimable Member
Joined: 3 months ago
Posts: 148
 

> It shouldn't page at 3 AM, but it should create a tracked incident.

So you're advocating for creating noise that doesn't require immediate action. That's just a Jira ticket with extra steps and overhead.

Our numbers: routing transitive criticals as low-urgency PD incidents created 47 tickets last month. Resolution time average? 18 days. It's a backlog tax.

If it can't be fixed tonight, it's not an incident. Use a dashboard or a weekly report. Don't clog the incident queue with work that belongs in a sprint.


show the math


   
ReplyQuote
(@jennifer2)
Eminent Member
Joined: 5 days ago
Posts: 19
 

Oh, excluding dev dependencies in the webhook logic is smart, I hadn't thought about that. Makes total sense.

> separate low-priority PagerDuty service for license alerts

Do you find that the separate service actually gets looked at, or does it just become a graveyard? We're using Slack for license stuff right now, but it's getting lost in the shuffle. 😅



   
ReplyQuote
(@jenniferl)
Trusted Member
Joined: 6 days ago
Posts: 31
 

I see your point about direct dependencies only, but what about transitive vulns in locked, version-pinned production dependencies? Those can be just as "in your control" to fix by updating the direct parent.

We tried that strict filter and missed a high-sev flaw in a transitive package because the parent lib had a strict semver pin. It took a dashboard scan two weeks later to catch it.

Maybe the filter should be "direct OR pinned transitive" instead of just direct?


Always testing the next best thing.


   
ReplyQuote