Skip to content
Notifications
Clear all

Troubleshooting: Deep Visibility logs missing on Linux servers.

5 Posts
5 Users
0 Reactions
2 Views
(@crm_hopper_2025_new)
Reputable Member
Joined: 2 months ago
Posts: 121
Topic starter   [#14950]

Alright, who else is running SentinelOne on their Linux estate and hitting the same wall? Every quarter I'm evaluating a new EDR or XDR platform, and while I grudgingly admit S1's prevention is solid, their reporting and visibility layer feels like a second-class citizen outside of Windows.

The specific gripe: Deep Visibility is enabled, the agent shows as healthy, but the logs for process creation, network events, or file operations on our Ubuntu 20.04/22.04 servers are just *missing*. The Singularity Data Lake either has huge gaps or shows nothing but heartbeat events. On Windows? Flood of data. On Linux? Crickets.

What I've already tried (and failed with):
- Verified the policy is assigned and Deep Visibility is toggled to "On" (not "Inherit").
- Agent is the latest S1 Linux bundle (22.4-ish).
- Reinstalled twice, once with the `--deep_visibility` flag during install, once without.
- Checked local agent logs – no obvious errors, just the usual noise.

The real kicker? Support's response was basically "sometimes the kernel module doesn't load correctly, check dmesg." Helpful. I'm starting to think this is less a troubleshooting issue and more a fundamental architecture problem—like they're leaning on eBPF but haven't got it consistent across kernel versions.

Has anyone actually gotten this to work reliably, or is this another case of a vendor checking the "Linux support" box while pouring 90% of their dev effort into the Windows agent? I'm already eyeing CrowdStrike again, but the data portability out of S1 is its own fresh nightmare.



   
Quote
(@devops_dad_v2)
Estimable Member
Joined: 4 months ago
Posts: 122
 

> "sometimes the kernel module doesn't load correctly, check dmesg."

Support's not wrong, but they're skipping the critical piece. The S1 kernel module on Linux is notoriously picky about secure boot, lockdown mode, and kernel version alignment. If you're running Ubuntu with a signed kernel or any custom modules loaded, the module can fail silently - agent shows healthy because the userland part runs fine, but the eBPF or kernel hook never attaches.

Couple things I've hit that fixed it for us:

- Check `lsmod | grep sentinel` - if it's not there, the module didn't load. Try `modprobe sentinel` manually and watch dmesg for "permission denied" or "operation not permitted" errors.
- On newer kernels, `CONFIG_LOCK_DOWN_KERNEL` or `CONFIG_SECURITY_LOCKDOWN_LSM` can block the module. If you're on a stock Ubuntu kernel, try the `-generic` variant instead of `-aws` or `-azure` which sometimes have lockdown enabled.
- If you're running containers or have `systemd` services with `ProtectKernelModules=yes`, the agent can't load its module. We had a few hosts where the agent was running under a systemd unit that blocked module loading.

Also, the `--deep_visibility` flag is a red herring - that just tells the installer to include the module, it doesn't force it to load. For container workloads, forget about Deep Visibility, the module won't attach to container namespaces. We switched to Falco for that layer and kept S1 for host-level prevention.



   
ReplyQuote
(@ginar)
Trusted Member
Joined: 7 days ago
Posts: 42
 

Right, the kernel module pickiness is real, but let's not let S1 off the hook so easily. The "agent shows healthy" while the core security function is dead silent is a classic vendor design flaw they've chosen not to fix.

The support script will have you chasing lockdown and kernel variants for hours. The real issue is their agent's health check is a lie. It confirms the service is running, not that it's actually collecting data. A proper agent would fail its own status check if the kernel hooks failed to load. They built it this way so the dashboard stays green while the actual protection evaporates.

You'll waste a week troubleshooting their fragile module when the root cause is a contract that doesn't penalize them for silent failures. Next renewal, make logging fidelity a service credit trigger.


Trust but verify.


   
ReplyQuote
(@jamesl)
Eminent Member
Joined: 1 week ago
Posts: 17
 

> "sometimes the kernel module doesn't load correctly, check dmesg."

The support script often misses this, but check `/var/lib/sentinelone/bin/` for the kernel module binary itself. We've seen cases where the download was corrupted or the installer placed an incompatible version. The userland agent can't probe for this, so it stays green.

If that's intact, the module may be blocked by the kernel's `integrity` subsystem, not just lockdown. Check `sudo cat /sys/kernel/security/integrity/ima/runtime_measurements | grep sentinel` for an entry. An absence there is a different failure mode.



   
ReplyQuote
(@amandak9)
Estimable Member
Joined: 1 week ago
Posts: 61
 

You've nailed the exact frustration that pushed me from evaluation to dropping S1 on our Linux fleet last year. The support deflection to kernel modules is real, but the architectural point is key.

Even when the module *does* load, we saw DV logs for common processes but a complete blind spot on containerized workloads. The eBPF hooks seemed to miss events inside pods. So you'd get logs for `systemd` but nothing from `kubelet` or the container runtime, which is where most of our risk lives.

Have you checked if the gaps correlate with a specific process type? It might not be all-or-nothing.


Show me the accuracy numbers.


   
ReplyQuote