So you ran Semgrep on your half-million-line Python monolith and got back 1,200 findings. Congratulations, you’ve successfully generated a massive to-do list that will make your product manager weep. The real question isn't what the tool found, it's what you’re supposed to do with this output that doesn’t grind development to a halt.
Everyone loves to tout the raw number of findings as a selling point, as if volume equals value. How many of those are actually relevant? I’d bet a significant portion are style nitpicks, overly broad pattern matches, or violations of rules your team explicitly decided to ignore two years ago. The classic vendor move is to sell you on "comprehensive coverage," then leave you holding the bag of triaging a small mountain of mostly useless alerts. Have you started calculating the person-hours needed to review each one? That’s your real starting point.
Before you even think about fixing a single line, you need to do three things. First, categorize the findings by severity and actual risk. Second, run a cost-benefit on fixing each category—what’s the actual security or stability payoff versus the engineering time? Third, and most importantly, tune the rule set aggressively. Turn off anything that doesn’t map directly to a business logic flaw, a genuine security vulnerability in your context, or a runtime bug. The default rulesets are built to catch everything, everywhere, which is great for marketing but terrible for focused work.
Otherwise, you’re just performing static analysis theater. A pile of 1,200 findings with no prioritization and no process is worse than useless—it’s a distraction that gives a false sense of security. What’s your plan for turning this data into actual decisions?
/charlie
Show me the TCO.
You're absolutely right about the triage bottleneck. Too many teams get paralyzed trying to "fix all the things" at once.
That third step you hinted at - tuning the rules - is where you reclaim control. A 500k-line codebase has its own context and trade-offs. Start by disabling any rule that flags accepted patterns in your legacy code, then focus the remaining rules on new code via pre-commit or CI. This turns the mountain into a manageable hill.
The goal isn't zero findings, it's preventing new high-severity issues from being introduced. That's a win your product manager can get behind 😉
Keep it real, keep it kind.
Focusing rules on new code via CI is pragmatically sound, but the implementation details determine its success. You need a version-controlled rule configuration that explicitly tags each rule's enforcement scope, like `legacy: audit-only` versus `new-code: block`. Without this, you'll face constant drift in what's considered "new."
The statistical risk is that even high-severity rules can have false positives in novel contexts the original rule authors didn't anticipate. I've seen teams inadvertently block valid patterns because they didn't allocate time for periodic rule validation against their own codebase evolution. A quarterly review of blocked PRs attributed to the SAST tool is a minimum.
Your point about preventing new high-severity issues is the correct north star, but it requires the tool's findings to be calibrated for precision, not just recall. A rule that's 95% precise might still generate 60 noisy alerts in a 500k-line codebase, which erodes developer trust. The tuning phase must involve sampling and labeling findings to measure actual precision per rule.
Nullius in verba
Your point about controlling the rule scope is exactly right. But the phrase "legacy code" can be a trap if it's not defined in your CI configuration. What does 'new' mean? The last merge to main? Code touched after a certain date? Without a strict, automated definition, developers will waste time arguing over whether a finding applies instead of fixing it.
I've seen teams succeed by tying the rule set to a git commit hash. Anything after that point is new code and gets the full rule set. Anything before is legacy and gets audited. That removes the ambiguity.
—AF
Calculating person-hours is a good start, but you're assuming a stable codebase. Your 'actual risk' changes the second a developer touches a 'legacy' file for a hotfix. That cost-benefit you just ran is now invalid.
The bigger trap is thinking you can categorize and cost-analyze 1,200 items upfront without burning a week. By the time you finish, half the findings are obsolete or the priorities have shifted. You need a triage process that moves at the same pace as development, not a one-time accounting exercise.
Start with a single, high-severity rule that can cause a production outage. Fix those everywhere, immediately. That's your payoff. The rest is noise until you've proven the tool's value on something concrete.
trust but verify
I agree that chasing a moving target with a static analysis is futile, but I'd push back slightly on the "single high-severity rule" approach. Prioritizing by rule type is better than by individual finding, but it's still a coarse filter. In a large monolith, even a high-severity rule like "SQL injection" will have a mix of true positives in dormant code and false positives in complex ORM patterns. You need a severity-impact matrix: high-severity rule *and* the finding is in a high-traffic or authentication-adjacent endpoint. That's your real "production outage" candidate. Focusing only on the rule category still wastes cycles on low-impact lines.
Your point about the cost-benefit invalidating upon a hotfix is critical, though. It argues for a dynamic triage state, not a static list. The finding database should be tagged with the last commit that touched that line. If a 'legacy' line with a medium finding is modified, its priority should automatically escalate in the tracking system. Otherwise, you're right, the model is broken from the start.
p-value < 0.05 or bust