Tag format mismatches are a classic vendor landmine. With Orca specifically, the default scope is usually based on their own internal cloud account onboarding, not instance tags. But if you've defined a custom scope rule, it's absolutely a black box.
The way I've forced visibility is to run a CLI query for all instances that *should* be in scope, dump their tags and IDs, then compare that list to the instances Orca actually reports on in its dashboard. The delta is your silently filtered set. It's manual, but it proves the point to support.
You mentioned a support ticket black hole. That's the real issue. Even when you get the "targeting logic" doc from them, it's often a high-level flowchart that doesn't specify the exact string parsing. I've had to escalate to a sales engineer to get the actual regex pattern used for tag key/value matching. It's exhausting.
You're hitting on the core frustration with these tools: the targeting logic is a configuration black box. That CLI delta check is exactly the right move. I'd take it one step further and formalize it as a pre-procurement validation step for any scanning vendor.
When we evaluate a new platform, we now demand a sandbox environment and run that exact delta test as part of the proof-of-concept. We hand them a CSV of instance IDs we expect to be scanned and require their platform to output a matching list. If there's a discrepancy, they have to explain the filtering rule on the spot. It turns a support mystery into a contractual deliverable.
It shouldn't take an escalation to a sales engineer to get a regex pattern. That's a red flag on their documentation maturity. Have you found any vendors that are actually transparent about this, or is it universally poor?
null
That pre-procurement validation is genius, and something more teams should adopt. It forces a concrete, testable requirement instead of vague promises.
> It shouldn't take an escalation to a sales engineer to get a regex pattern.
Totally. In my experience, the vendors with decent transparency are the ones who've been burned by a major deployment failure and had to rebuild trust. I've seen some newer, niche players be more open because they're trying to differentiate. The big established names? Almost universally poor, treating the targeting engine as "secret sauce" instead of a configurable component.
One caveat with your CSV method: watch for dynamic scoping based on runtime state. We had a case where our static list matched, but their platform later filtered instances where a specific process was running, which we only caught because scans became inconsistent. So the validation needs to cover both the initial inventory *and* the runtime filter logic, which is even harder to get out of them.
pipeline all the things