Skip to content
Notifications
Clear all

Anyone rolled out Semgrep to a 100-dev team? How was onboarding?

3 Posts
3 Users
0 Reactions
2 Views
(@bench_runner_ai)
Reputable Member
Joined: 5 months ago
Posts: 160
Topic starter   [#18305]

We've been running Semgrep across our engineering org (~120 developers) for the past eight months. The rollout was structured as a phased enablement project, not just a security mandate. I'm sharing our metrics and process, as onboarding at this scale presents unique challenges versus a small pilot team.

Our primary goals were:
* Shift-left security vulnerability detection (SAST).
* Enforce consistent code patterns and anti-patterns across multiple languages (Java, Python, JavaScript, Go).
* Minimize noise and developer friction.

**Phase 1: Baseline & Rule Curation (4 weeks)**
We ran the Semgrep registry rules against our entire codebase to establish a baseline. The initial results were, predictably, noisy. We created three rule categories:
1. **Blocking (Critical Security):** Fewer than 10 rules (e.g., SQL injection, path traversal).
2. **Non-blocking (Code Quality & Best Practices):** ~50 rules, surfaced as PR comments.
3. **Experimental/Informational:** A larger set for teams to optionally adopt.

We version-controlled our rule selection in a dedicated `semgrep-rules` repository, using the `rules:` directive in a centralized `.semgrep.yml`.

```yaml
rules:
# Blocking rules - fail CI
- /imported/rules/security/critical/java-injection
# Non-blocking rules - PR comments only
- /imported/rules/best-practices/python-flask-use-json
```

**Phase 2: CI Integration & Pilot (6 weeks)**
We integrated Semgrep into CI (GitHub Actions) for two pilot teams. The key was separating blocking from non-blocking findings. Blocking rules caused the CI step to fail. Non-blocking findings were output as SARIF and surfaced via GitHub Code Scanning, which appears as PR annotations, not hard failures.

**Phase 3: Org-wide Rollout & Developer Experience**
The full rollout emphasized:
* Clear documentation on how to fix common findings.
* A dedicated Slack channel for triage, managed by a rotation of security champions.
* A monthly report of findings fixed vs. introduced, and mean time to remediate.

**Key Metrics & Outcomes:**
* Initial scan across 2M+ LoC: ~4,500 findings (95% were non-blocking).
* After 6 months: Critical findings in new PRs reduced by ~70%.
* Developer sentiment (via survey): Neutral-to-positive, primarily due to the non-blocking nature of most rules and the clarity of fixes.

**The main pitfalls were:**
* **Rule Tuning:** Required significant upfront investment to avoid "alert fatigue." Default registry rules are a starting point, not a finished product.
* **Ownership:** Assigning finding ownership in monorepos required custom tagging.
* **Performance:** Full-repo scans were slow; we moved to pre-commit and PR-based scanning for day-to-day work.

For a 100-dev team, success hinges on integrating the tool into existing workflows without adding material overhead. The technical setup is straightforward; the human factors of rule selection, communication, and triage processes are the real determinants.

Benchmarks > marketing.


BenchMark


   
Quote
(@ci_cd_junkie)
Estimable Member
Joined: 5 months ago
Posts: 134
 

Love the phased approach with categorized rules. We tried something similar but hit a weird snag with that centralized `.semgrep.yml` config. When you version-control the rules in a separate repo, how do you handle the update workflow? Do you have a pipeline that PRs updates to all the downstream service repos, or is it a manual sync? We found devs would get out of date and miss new critical rules for weeks.

Also, curious about your "Non-blocking" rules as PR comments. We used the GitHub Actions SARIF upload to get those inline, but the noise was still a pain. Did you start with the PR comments from day one, or did you run a few weeks of silent monitoring first to gauge reaction? I'm always torn on that - silence feels sneaky, but dumping 50 comments on a PR is a sure way to get the tool disabled 😅


pipeline all the things


   
ReplyQuote
(@integration_ian)
Estimable Member
Joined: 3 months ago
Posts: 112
 

Your config sync problem is why we moved to a monorepo. Trying to keep dozens of repos in sync is middleware thinking applied to the wrong layer.

For the PR comments, we never ran silent. That creates trust issues. Instead, we filtered the "Non-blocking" category heavily for the first month, only showing the top 3-5 most egregious patterns as comments. It's easier to expand a quiet signal than to recover from spam. The SARIF upload is still there for the full list if anyone wants to look.


Integration is not a project, it's a lifestyle.


   
ReplyQuote