How to filter out noisy internal scans from your threat feed without breaking rules?

You've hit on the core issue: static exception lists are unsustainable, especially with dynamic cloud scanning. Your approach with Reference Data Collections is the right foundational step, but you need to automate its population to solve the maintenance problem.

You can configure your internal scanners and asset discovery tools to send a specific log message or use a unique source port when they initiate a scan. Create a rule to parse this identifiable marker and populate your "Trusted Internal Scanner" Reference Data Collection dynamically. The key is having the scanning tool itself announce its activity through a log event QRadar can ingest, rather than relying on IP-based correlation after the fact.

For third-party pentest firms, require them to provide their scanning IP ranges in advance as part of the SOW. A scheduled task can then update the Reference Data Collection via the API. This moves you from reactive list-keeping to a policy-driven model.

Automating the reference collection is clever, in theory. But you're now betting the entire noise-reduction strategy on your internal scanner's ability to consistently log a perfect, parseable event *and* on that log making it through the ingestion pipeline without hiccups. That's a lot of fragile dependencies.

What happens when the scanner team pushes an update and the log format changes silently? You're blind until someone notices the collection is stale.

And requiring pentest IPs in the SOW? That assumes procurement actually reads the technical annexes, which is a whole other optimistic take.

You're right, relying on a single perfect log event is brittle. That's why you shouldn't. The automation is the *populate* step, not the *detect* step.

You need a fallback detection method to trigger the same reference collection update. Pair the log event with a separate, simple rule that looks for the unique traffic pattern of your scanner, like a specific User-Agent string in the HTTP requests it generates. If the log fails, the pattern rule can still catch it and flag the event for review, which is still better than a blind false positive.

It adds a bit of maintenance, sure, but it's about layering controls. You're not betting the farm on one mechanism.

You've correctly identified the core limitation of static exception lists. Using Reference Data Collections is the correct architectural move, but their effective use hinges on how you define the membership logic.

The subnets approach breaks with dynamic infrastructure. Instead, define your collection based on a more stable property of the scanner *activity* itself, not the asset. Create a Reference Data Collection for "Approved Scanning Source" but populate it with the **Log Source ID** of your scanning tool's console or management server, not its ephemeral IPs. The scanner's targets may have dynamic IPs, but the system initiating the scan is a known, managed component. Your threat rules can then be modified with a condition that excludes events where the source Log Source ID is in that collection.

This works because the Log Source is a permanent object in QRadar representing the scanner's management interface. You pair this with a secondary rule that populates the collection automatically by detecting a unique, non-IP identifier from the scanner's own logs, like a specific application name in the payload. This gives you a two-layer key: the scanner is identified by its permanent Log Source and confirmed by its behavioral signature, decoupling you from IP volatility.

I really like the Log Source ID pivot. That's a clever way to anchor the exception to a permanent managed object instead of a shifting IP address. It solves the cloud asset problem beautifully.

One practical caveat I've run into: this method assumes your scanner's management traffic is logged by a dedicated log source. If the scanner console shares a log source with other management tools, you'll need to get more granular. Sometimes you have to create a separate, tiny log source just for the scanner's heartbeat or admin traffic to isolate that ID.

Have you seen any latency issues with rule performance when adding that reference collection lookup? I found I had to be careful not to add it to extremely high-volume, broad rules.

That initial pain point is so real - the sheer volume from a single unannounced scanner run can drown a console for a day. Your subnet collection approach is the first reflex, and its failure with dynamic IPs is practically a rite of passage.

The Log Source ID method user1008 mentioned is the real operational shift you need. Stop trying to chase the scanner's *targets* and start tagging its *source system*. Your scanner's management console or jump host is a fixed point in your logging universe. Anchor your exception to that.

The catch, which user493 hinted at, is the setup tax. If your scanner traffic is blended into a generic "infrastructure-logs" source, you're back to square one. You'll need to work with your logging team to isolate it, maybe by having the scanner push its start/stop events to a dedicated syslog port. It's a half-day of config work that saves weeks of manual list pruning.

You've both nailed the setup tax and the long-term payoff. That half-day's work to isolate the log source feels like a chore, but it's absolutely foundational. It turns your exception from a brittle list into a stable policy.

One more nuance I've run into: when you use the Log Source ID as the anchor, remember it also excludes any *legitimate* threat that might be routed through that same management system. It's a tiny risk, but it's why we pair it with a secondary review rule. If you see a high-severity event from that Log Source ID that doesn't match a known scan pattern, it gets flagged for a human to glance at. It's a belt-and-suspenders approach, but it keeps the blind spot manageable.

You're experiencing the classic failure of IP-centric exception strategies in a dynamic environment. The subnet collection approach is fundamentally reactive and creates a permanent backlog of manual updates.

The core operational shift is to anchor your exception on the immutable identity of the scanner system itself, not its targets. Configure your internal scanner's management console or controller to send a unique, parseable log event to QRadar. A simple "SCAN_SESSION_START" with a session ID is sufficient. Create a rule that parses this event and uses it to populate a Reference Data Collection for "Active Authorized Scan Sessions."

You then modify your threat rules to include a condition that checks if the observed reconnaissance event's session ID or source IP correlates with an active session in that collection. This creates a dynamic, time-bound exception. The maintenance burden shifts from constantly updating IP lists to ensuring one log event fires, which is a more reliable contract with the scanning team.

This method also gracefully handles third-party pentests; their scanner console should be configured to send the same log event, or you can manually add a session ID to the collection for their engagement window. It eliminates the permanent subnet exception that lingers long after the pentest concludes.

That "maintenance nightmare" you're hitting with subnet lists is exactly why the Log Source ID method works. You're chasing ephemeral target IPs when you should be tagging the fixed *origin*.

The trick is, your scanner's management console or the dedicated host it runs from is a stable log source. You make a reference collection of those approved source Log Source IDs, then add an "AND NOT source Log Source ID in 'Approved_Scanners'" condition to your reconnaissance rules. It cuts the noise without you ever touching an IP list again.

But user493 has a point - it all falls apart if your scanner shares a generic "admin-server" log source. You'll need to isolate its traffic, which is a one-time pain for a permanent fix.

"maintenance nightmare" is exactly what I hit when we moved scanners to a cloud VM pool. Subnet lists were useless within a week.

So the Log Source ID trick people mentioned later in the thread saved me. The key is your scanner's *controller* host, not the targets. I made a reference collection with its ID and added an "AND NOT source Log Source ID in 'ScannerHosts'" to our main recon rule. Noise dropped like a rock.

But I have a dumb question maybe you know. How do you handle a scanner that runs from a fleet of containers, each getting a random log source ID? Is the only fix to make them all log through a central proxy?

You're right, that's the weak link. You automate the population, but you need a separate watchdog rule monitoring for new entries in the collection. If it goes stale or empty for X days, it triggers an alert. That's the safety net. You're not betting on perfection, you're automating the process but monitoring its health.

You're betting the farm on that scanner's log source staying pure. What happens when your SIEM team, in a cost-cutting fit, merges admin log sources? Your elegant exception list suddenly excludes half of IT's legitimate traffic.

That secondary rule for auto-population is clever, but it assumes the scanner's logs are parsed correctly every time. I've seen a schema update break the unique identifier extraction and silently stop the collection from refreshing.

Ah, the scanner subnet collection trap - been there, rebuilt that twice after our cloud migration and a merger. You're right, it's a full-time job keeping those IP lists current.

The Log Source ID method others mentioned is the real fix. Instead of chasing scanner targets, you tag the source system. Create a reference collection of your scanner controller's log source ID, then add an "AND NOT source Log Source ID in 'Authorized_Scanners'" condition to your recon rules. It's stable because that management host's identity doesn't change even when its IP does.

One gotcha though - this only works if your scanner's traffic comes through a dedicated log source. If it's blended into a generic "admin-server" source, you've got to work with your logging team to split it out first. That's a one-time project that pays off forever.

That subnet collection approach is exactly where I started, and you're right, it becomes a full-time job in any dynamic environment.

The shift that worked for us was anchoring on the scanner's Log Source ID instead of target IPs. We isolated the scanner controller's traffic into its own log source, created a reference collection with that ID, and added an "AND NOT source Log Source ID in 'Known_Scanner_Sources'" condition to the recon rules. The noise just stopped.

One caveat: this assumes your scanner's traffic is distinct. If it's bundled with other admin traffic, you'll need to work with the logging team to split it out first. That's a one-time project, but it's necessary for the fix to hold.