Skip to content
Notifications
Clear all

How do I handle findings that are intentional (like test data)?

2 Posts
2 Users
0 Reactions
0 Views
(@integration_ian_2)
Reputable Member
Joined: 2 months ago
Posts: 159
Topic starter   [#17572]

Hey everyone,

I've been deep-diving into Semgrep for the last few months, integrating it into our CI/CD pipelines to catch security issues and code smells before they hit production. It's been a game-changer for our team's code quality, but I keep running into the same friction point: managing findings that we've **intentionally** decided are acceptable.

For example, we have test files and fixtures that contain what Semgrep rules flag as "secrets" or "hardcoded passwords." These are obviously just dummy data for our integration tests, like `password="test123"` or an AWS key that's clearly a placeholder. Similarly, we sometimes have to suppress a specific linting rule in a particular module because of a legitimate technical constraint.

The problem is, these intentional findings clutter up the output, creating noise that can cause real issues to be missed. I'm trying to find a clean, scalable way to handle this without just turning off entire rules or creating a maintenance nightmare.

Here’s a sample of what I'm dealing with:

```yaml
rules:
- id: hardcoded-secret
pattern: passwords*=s*"$PASS"
message: Hardcoded secret detected
severity: ERROR
```

And a test file triggers it:
```python
# test_integration.py
def test_auth_flow():
# This is intentional test data, not a real secret
test_password = "superSecretPassword123"
...
```

So far, I've explored a few workarounds, but each has its downsides:

* **Using `nosemgrep` comments:** This works but feels a bit dirty sprinkling them everywhere in test files, and it's a line-by-line management headache.
* **Path exclusions in the rule itself:** Adding `exclude: "tests/"` to a rule is clean, but then we lose coverage for actual secrets that might *accidentally* end up in test code.
* **Separate, less strict policies for test directories:** This is my current leaning—running a different set of rules or a tuned-up policy against `tests/` paths. It adds complexity to the pipeline config though.
* **Baseline files (`.semgrepignore`):** This seems promising for accepting current findings, but I'm worried it'll become a "set it and forget it" dump where new, legitimate issues get ignored.

I'm curious how others in the community are architecting their setups. Specifically:

* Do you maintain separate scanning policies for application code vs. test code?
* Have you built any automation around generating or managing baseline files to keep them lean?
* Are you using the `--disable-nosemgrep` flag in CI for certain paths to ensure the comments aren't abused in production code?
* Any clever Makefile or GitHub Action workflows that handle this bifurcation cleanly?

I love the control Semgrep gives us, but I want to make sure we're not drowning in false positives. The goal is a pipeline where a new finding is a genuine event that needs review.

api first


api first


   
Quote
(@henry)
Estimable Member
Joined: 1 week ago
Posts: 79
 

Totally feel your pain. That noise is exactly what makes teams start ignoring the reports. Have you looked at Semgrep's inline suppressions using `nosem` comments? You can tag a finding with a specific rule ID right next to the line, which keeps the suppression tightly scoped.

One caveat from our setup - you need to be disciplined with it. If you just `# nosem` without the rule ID, you might accidentally silence a new, relevant rule later. Also, for whole test directories, we use path-based rule exclusions in our `semgrep.yml` config. It's cleaner than per-file comments but you have to keep that config updated when you add new test suites.


Cheers, Henry


   
ReplyQuote