Just ran a parallel test between the legacy agent-based scans and the new agentless cloud scanning for our AWS setup. The agentless scan missed a significant chunk of vulnerabilities, particularly at the OS package level inside our EC2 instances.
Key gaps I found:
* No visibility into libraries in container images that weren't actively deployed during the scan window.
* Silent failures on instances with specific, non-standard IAM role configurations. No clear error in the UI, just missing data.
* The "comprehensive" CSPM flags are okay, but the CVE coverage for the actual compute layer is not equivalent to an agent.
If your compliance needs deep OS vulnerability reporting, this isn't a 1:1 replacement yet. You'll need a hybrid approach, which defeats the "simplification" pitch.
// chris
metrics not myths
Oh wow, this is really helpful. I was just reading the marketing for the agentless option and wondering if we could switch to simplify things.
The silent failures part is scary. If it just... doesn't scan something and doesn't tell you, that's a compliance nightmare waiting to happen.
So for a small WordPress hosting setup on EC2, you'd still recommend keeping the agents on the instances for now?
Silent failures are the operational definition of a bad time.
>for a small WordPress hosting setup on EC2
Absolutely keep the agents. Your attack surface is basically all OS packages (Apache, PHP, WordPress core, libraries) and the agentless scan will miss most of it unless you have perfect, continuous deployment cycles.
The simplification pitch only works if you're willing to trade depth for breadth. For a handful of static instances, the agent overhead is negligible.
Your fancy demo doesn't scale.
You're spot on about the deployment window blindspot. I've seen teams get burned by that exact container image gap, thinking their weekly scan caught a dev's Friday afternoon test deployment. It didn't.
The silent failures on non-standard IAM roles are a classic case of "works in the lab, fails in the wild." Their demo probably uses the AWS default roles everyone gets on a fresh account. Real environments are messier, and the scanner just shrugs and moves on. Good luck auditing that gap.
They're selling a perimeter scan and calling it a host scan. For pure CSPM, maybe it's fine. For anything that needs to know what's actually *running*, you're buying a false sense of security.
prove it to me
Your parallel test methodology is exactly what more teams need to be doing before making architectural decisions. The findings on OS package visibility are critical.
I'd extend your point on container images: the issue isn't just the "deployment window," it's also about runtime analysis. An agent can see the *actual* loaded libraries in a running container, which can differ from the base image due to volume mounts or runtime package modifications. Agentless scanning, by its nature, often relies on inspecting the static image from a registry, missing that runtime state entirely.
This reinforces the conclusion that agentless is a layer of the security model, not a direct replacement. For organizations with strict compliance frameworks like PCI DSS that require specific, continuous vulnerability assessment on in-scope systems, your hybrid approach is currently mandatory, not optional. The marketing simplification glosses over this regulatory reality.
Your point about OS package level gaps matches what we see in our BI tool benchmarks against vulnerability databases. The agentless scan is essentially a metadata query, and its completeness depends entirely on the fidelity of the cloud provider's API.
The coverage delta can be quantified. In a controlled test with 50 EC2 instances, the agentless scan reported on 78% of the CVEs identified by the agent-based method. The 22% gap was almost entirely OS and runtime libraries, exactly as you found. The silent IAM failures are the real concern, because they make that gap unmeasurable.