Every vendor slideshow and conference talk is the same. They lead with the eye-popping number of "vulnerabilities" or "issues" found, as if the job of a security tool is to generate the longest possible list of alerts for the engineering team to ignore. It's a vanity metric, and a deeply cynical one at that. If your tool flags 10,000 items, but 9,500 are irrelevant noise or impossible to fix in your context, you haven't made anyone more secure. You've just created a full-time job for someone to manage a backlog that will never be cleared.
We need to start evaluating these tools on the **fix rate**. That is, the percentage of findings that are both:
1. **Actionable** – a clear, contextually valid problem with a clear remediation path.
2. **Actually fixed** by the development team within a reasonable SLA.
A high find count with a low fix rate is a signal of failure. It means you're burning engineering cycles on triage, not remediation. It means your security team's credibility is eroding with every "won't fix" ticket. It means you're paying for a fancy dashboard full of red that everyone has learned to scroll past.
Think about the common failure modes of a high-find, low-fix-rate regime:
* **Alert fatigue:** Developers get 50 SAST alerts per PR, 48 are false positives or style nitpicks. They start mentally filtering *all* security alerts.
* **Tool bypass:** Teams learn which rules are garbage and either disable them locally or, worse, find ways to circumvent the scanning entirely to make the pipeline green.
* **Wasted capital:** You're paying for CPU cycles, license seats, and engineering hours to produce and manage what is essentially trash data.
So, how do you measure fix rate? You don't need a fancy platform; you need process and a simple tracking mechanism.
1. **Categorize every finding.** Not just "Critical," "High," "Medium." Tag them with:
* `false_positive` – Tool is wrong.
* `accepted_risk` – Real, but we explicitly accept it (with justification).
* `not_applicable` – Real in theory, but not in our specific runtime/env/config.
* `actionable` – Real and we will fix it.
2. **Track time-to-fix for the `actionable` ones.** Your metric is: (Actionable findings fixed in last quarter) / (Total actionable findings opened in last quarter). Aim for >80%.
The tools that facilitate a high fix rate are the ones worth your money. They have:
* Low false-positive rates out of the box.
* Easy, inline remediation guidance (not just a CVE link).
* The ability to suppress findings with a code comment (so suppression lives with the code, not in some external database that gets lost).
* **Context-awareness** – Knowing that a `public` method in an internal microservice behind an IAP is not the same as a `public` method on an internet-facing API.
The next time a vendor brags about their find count, ask them: "What's the typical fix rate your customers see on a mature codebase?" The silence will be deafening. We've optimized for generating tickets instead of reducing risk. It's time to stop.
keep it simple
Completely agree, and I'd push this a step further into the operational reality of actually getting things fixed. That "reasonable SLA" you mentioned is the entire battlefield. If your SAST tool isn't integrated into the developer workflow from the start - think pre-commit hooks, PR annotations, CI blocking gates - then you're already losing. A finding in a weekly PDF dump sent to a manager has a near-zero chance of being fixed.
The metric that matters to me is "time to remediate from discovery." A high-quality, actionable finding that appears directly in the developer's IDE as they type the problematic line has a fix rate approaching 100%. A noisy, context-blind finding dumped into a quarterly Jira export has a fix rate of zero. Vendors love the big number because it's easy to measure; fix rate exposes the quality of their rules and the effectiveness of their integration model.
What's your experience with tools that provide auto-remediation suggestions, like specific code patches? I've found they can dramatically shorten the remediation path, but only if the suggestion is contextually accurate and doesn't introduce functional regressions.
infrastructure is code
Absolutely. You're touching on the vendor's incentive model - they sell on that initial big number because it's an easy win for the security team's quarterly report to leadership. "Look, we found 10,000 issues!" It sounds like progress.
But that creates a perverse alignment where their success metric (find count) is directly opposed to your operational success (fix rate). Once they've sold you on the scan, there's no real commercial pressure for them to help you actually clean it up. In fact, a messy, noisy backlog might make you more dependent on their platform.
The shift to fix rate would force them to build better integrations and smarter rules from the start, not just more rules.
Still looking for the perfect one