Good point about existing sessions. Mobile apps are notorious for this - they'll cache an old authentication grant and skip anything new, like a fresh ToU. It's like they live in their own little world.
But the whole "test group is too small to call it a bug" line? Nah. 20% of a targeted group failing is a significant pattern, not an edge case. That's enough to raise a proper support ticket, not just shrug it off as config weirdness. Microsoft's own "works on my machine" logic shouldn't fly.
Your session theory is solid for the 'why', though. I'd add: check if those three users are frequent mobile/desktop app users, while the others are mainly browser-based. That'd be the smoking gun.
Trust but verify.
Everyone's fixated on sign-in logs, but the logs won't show you a missing assignment at the policy engine level. I've seen this exact pattern.
You said "no conflicting CA policies," but that's from your view in the portal. The system evaluates them in a specific order, and a grant control from a separate, lower-priority policy can satisfy the requirement before your ToU even gets a chance to fire. The user sees it as a bypass.
Try this: create a test CA policy that *only* requires the ToU for the same app and assign it directly to one of the three users. If they still skip it, then you've got a platform bug. If they don't, you've got a policy interaction problem in your main setup.
-- bb
That's a great isolation test. I've used a similar approach before, but it only works if you clear all existing sessions first. A user with a cached session from a prior compliant login might still bypass your new test policy, making you think it's a bug when it's actually old state.
The policy order point is spot on, and it's a headache because the portal doesn't show you that evaluation sequence. Makes you wish for a proper simulation tool.
So the 3 users who bypass it completely, are they all using mobile apps? I had a similar glitch where cached grants in Teams mobile skipped new ToU prompts, even after clearing browser sessions. Might explain why it's only a subset.
Trying to figure it out.