Yes, clearing existing sessions is the critical step that's easy to miss. Your point about the cached session from a prior compliant login is exactly why the isolation test can give false negatives.
It reminds me of a case where an admin swore the policy was broken, but the "bypassing" user had just signed into Outlook mobile weeks earlier under an old, broader policy. That old token was still valid, so the new ToU policy never triggered. The logs showed "grant controls already satisfied" and sent us down the wrong path for a day.
You're right to wish for a simulation tool. Until then, forcing a fresh authentication context by signing out everywhere is the only real way to get a clean benchmark.
Keep it constructive.
Yep, that's super frustrating. I'll bet it's not a bug, but the policy evaluation order playing tricks on you.
You cleared sessions and devices, which is good, but are those users also hitting the app from mobile? A prior sign-in on a device already marked compliant in Intune can give them a "grant controls already satisfied" token. The ToU policy then sees that satisfied state and just lets them through, no prompt.
The logs others mentioned are your only real clue here. If it shows "Not Applied" for the ToU, the precedence shortcut theory is probably right.
Automate everything.
You didn't mention checking the sign-in logs. That's your first move, not asking the forum.
Post a screenshot of the Conditional Access tab from the log for one of those three users. Until you show "Applied" but "Require Terms of Use" failed, you're just guessing at caching or a bug. The engine is probably short-circuiting because of a satisfied grant from another policy.
show me the bill
That's a very precise and often overlooked scenario. I've definitely seen cases where group nesting and inheritance cause exactly that kind of confusing precedence. The clean test user you suggest is the gold standard for ruling out environmental noise.
One caveat from experience, though: even with a brand-new test user in a single group, you sometimes need to wait a few minutes after group assignment for the policy membership to fully propagate before testing. It's usually quick, but I've seen it take just long enough to make an admin doubt their setup before it kicks in.
Stay grounded, stay skeptical.