This point about manual vs learned suppression hits home. I was testing a tool last month and noticed the same plateau - they'd just added file-level ignores instead of recognizing our internal validation wrapper.
> If week two shows a plateau...that's not the engine learning your logic
That's exactly it. Should we be asking vendors for the ratio of "path-based suppressions" to "rule-based suppressions" generated during the PoC? Might show if the tuning is actually intelligent.
Yes, asking for that suppression ratio is a solid, concrete metric to request. It exposes the tuning mechanism. A tool that's truly learning should show a shift from path-based ignores towards rule-based suppressions over the trial period.
However, be cautious about how the vendor defines a "rule-based suppression." I've seen them classify moving a file-level ignore to a directory-level ignore as "rule-based," because it uses a wildcard pattern, even though it's still just ignoring a location, not understanding code semantics. You need to see the rules targeting specific sink or validation patterns, not just broader path exclusions.
A related question is whether the vendor's engine can propagate a rule-based suppression automatically. If you tag an internal wrapper as a sanitizer, does it apply that understanding to future scans without manual intervention for each new finding? That's where the long-term maintenance cost separates the tools.
Great point about the suppression ratio. It reminds me of when we were tuning CloudTrail alerts and the system kept ignoring entire S3 bucket paths instead of learning the specific IAM policy pattern. We had to dig into the logs to see if it was actually learning.
That automatic propagation you mentioned is key for long-term cost. If you're constantly rebuilding suppressions after every deployment, the hidden operational cost eats any license savings.