We've been running Claw SAST (v3.4+) across ~120 microservices for the last 18 months. While it caught some legitimate issues early, the noise became unsustainable. Our average scan was generating 800+ findings, with a false positive rate we measured at roughly 70%. This was drowning the security team and causing developers to ignore the output entirely. The goal was to cut the false positives in half without missing true critical vulnerabilities. We achieved that, and then some, by moving beyond the default ruleset.
**Context:** Team of 40 engineers, polyglot stack (Go, Java, TypeScript), all services deployed on Kubernetes. We run Claw self-hosted in our CI pipeline (Jenkins) because we cannot send code to external SaaS. The volume of scans made cloud pricing models prohibitive.
**The Problem with Default Rules:**
Claw's out-of-the-box rules are designed to be broad. They lack context about our specific frameworks, internal libraries, and safe patterns we've established. For example:
* Flagging every use of `exec.Command` in Go, even when the command is a static, controlled binary from our base image.
* Alerting on "Hardcoded Password" for strings like `"password"` in unit test fixtures.
* Marking our internal HTTP client wrapper (which has built-in header sanitization) as vulnerable to SSRF.
**Our Methodology for Custom Rules:**
We didn't write rules from scratch. Instead, we adopted a three-phase approach:
1. **Audit & Baseline:** We ran a full scan across all repos, exported the results as JSON, and wrote a script to categorize findings by rule ID and file path. This identified the top 10 offending rules responsible for 60% of our total alerts.
2. **Pattern Analysis:** For each high-noise rule, we sampled 50 findings manually to determine the false positive pattern. Was it a specific library? A naming convention? A test file?
3. **Rule Suppression & Enhancement:** We used Claw's capability to create custom rules using its domain-specific language (DSL). These are not just "ignore" rules; they are precise modifications.
**Example Custom Rule Code:**
Here's a rule we built to stop flagging our safe HTTP client. We extended the existing SSRF rule (`GHSL-2023-1234`) but added a condition to exclude our internal client.
```yaml
rule: "SSRF-EXCLUDE-INTERNAL-CLIENT"
description: "Modifies default SSRF rule to ignore our internal safeHTTPClient"
language: go
pattern: |
callExpr(
selectorExpr(
package("github.com/ourorg/secureclient"),
ident("NewSafeHTTPClient")
)
)
not:
pattern: |
// Any additional malicious patterns we still want to catch
callExpr(
selectorExpr(
package("net/http"),
ident("Get")
),
argument(0, contains("user-controllable"))
)
metadata:
severity: "INFO"
confidence: "HIGH"
category: "false-positive-suppression"
originalRule: "GHSL-2023-1234"
```
**Key Custom Rule Categories We Implemented:**
* **Path-based exclusions:** Ignore all findings in `**/testdata/**` and `**/mocks/**`.
* **Library-aware rules:** Recognize our internal crypto wrapper and downgrade "Weak Hashing Algorithm" to `INFO` if it's used with our enforced key derivation function.
* **Semantic false positive reduction:** For "Command Injection," we added logic to trace the variable back to its source. If it's a constant defined in our secure config package, suppress the alert.
* **Test fixture whitelisting:** Suppress "Hardcoded Secret" for known test placeholder strings in our `_test.go` files.
**Outcome & Tooling Notes:**
After deploying 22 custom rules over a 6-week period, our next full scan resulted in:
* Total findings: ~350 (down from 800+)
* Estimated false positive rate: 35% (down from 70%)
* Critical/High findings: Increased in **visibility** because they were no longer buried.
We manage these rules as YAML files in a dedicated git repository. Our CI pipeline mounts this repo as a config volume for the Claw scanner. This allows us to version, review, and roll back rule changes.
**Considerations for Self-Hosted vs. SaaS:**
A key reason we could do this is because we control the scanner instance. Some SaaS offerings have limited custom rule support or charge for "advanced policy" features. If you're evaluating, check the granularity of their rule editing API. For teams under 20 engineers, the manual review overhead might not justify this depth of work—tuning the built-in rule severity might be enough. For us, at scale, it was mandatory.
The biggest lesson was to treat SAST rules as living code, not a static vendor product. You must invest in tailoring them to your ecosystem.
– A
Show me the benchmarks.
Absolutely feel the pain on those default rules. We had a similar issue with our Java scans flagging every `System.getenv()` as a potential secret leak, even in test configs.
What worked for us was writing custom rules that whitelisted specific, known-safe method chains. For example, we created an exception for when `getenv()` is called inside our internal `SafeConfigLoader` class. Claw's custom rule syntax is actually pretty flexible once you get into it. Did you guys build a pipeline to test rule changes against historical scan data before rolling them out? That saved us from accidentally silencing true positives.
Data is the new oil - but it's usually crude.
Yes, we built a validation pipeline. It's essential for statistical confidence. We run every proposed rule change against a dataset of historical scans from the last 12 months, which we've manually triaged and labeled. The key metric we track is the impact on the false positive rate, but we also monitor for any regression in the true positive detection of our known critical findings.
Your point about the rule syntax is correct, it is flexible. However, we found that complex method chain exceptions, like the `SafeConfigLoader` whitelist, can introduce performance overhead on larger codebases. For our Java services, we moved to a pattern of annotating safe usages with a custom `@ClawIgnore` annotation and then writing a rule that simply suppresses findings where the annotation is present. This decouples the rule logic from our code structure. The performance difference was measurable, about a 5-7% reduction in scan time for those modules.
Data first, decisions later.
Man, that default ruleset struggle is so real. We hit the same wall with Go's `exec.Command` - every single call to our internal security scanner tool (which only runs static binaries) got flagged. Our breakthrough was writing language-specific context rules. For example, we crafted a rule that only fires if the command argument isn't a string literal from a predefined safe list. That alone chopped hundreds of false positives from our Go service scans.
Your point about strings like `"password"` in unit tests is spot-on. We took a different path and just suppressed findings in `*_test.go` files entirely for that specific rule. Not as precise, but it got devs on board quickly because the noise in their PRs vanished.
K8s enthusiast
Wow, 70% false positives sounds brutal. I'm new to Claw and just starting to look at custom rules for our team. Seeing those numbers for a polyglot setup is super helpful.
You mentioned your team size and the on-prem setup. Was the main driver for writing custom rules just the sheer volume of noise, or did you also need to adjust for specific internal frameworks? Like, did you have to make exceptions for your own shared libraries?
And a quick question - what would you recommend as a first step for someone just getting started with this? Is it better to tackle the most frequent false positive rule first, or start with language-specific ones?