Agree completely. More bad data just creates more work.
Your point about intent is the real operational cost. A high capture rate of "zero result" searches from bots forces my team to waste cycles investigating product gaps that don't exist. The metric becomes noise.
The partial query and autocorrect issue you mention is another layer. Server logs see the raw keystrokes, but they miss the client-side context of a user backspacing and correcting. Client-side tools see the final input, but might miss the abandoned partials. Neither gives you true user intent, just another incomplete signal.
Five nines? Prove it.
The whole "but did you baseline against actual server logs?" argument is a bit of a trap. It implies server logs are some pristine oracle of truth, when they're just a different, equally messy data source obscured by load balancers, CDN caches, and bot traffic. They're not a baseline, they're a conflicting opinion.
What you're really asking for is a third, also-imperfect, dataset to triangulate against. That's useful, but it doesn't solve the core problem. You're just layering more "different types of inaccurate" on top of each other, hoping the blur averages out to something clear. It usually doesn't, it just gives you three numbers to argue about instead of two.
The real question isn't which source is more accurate, it's which source's inaccuracies are more predictable and correctable for your specific stack. A 9% undercount you can model and adjust for is infinitely more valuable than a 'complete' log full of noise you can't decontaminate.
Thanks for sharing the test results, this is really helpful for someone like me trying to evaluate these tools. I'm curious about something in your snippet though. You mention the gap is especially big for SPAs. How much of that 18% undercount for Whitebox was due to the SPA behavior versus other issues? Like, did you see a big difference in capture rates between a traditional page load site and the SPA in your tests?
CloudNewbie