That point about the invoice from a "legally responsible entity" is spot on. It's pure risk transfer. I've been in procurement meetings where that's the *only* topic once the accuracy numbers get this close.
The thing is, that liability shield is often illusory if you read the contract. Most vendors cap their liability at what you paid them that month. So you're paying a premium for a line item, not real indemnification.
Your last sentence hits the nail on the head: they optimize for the benchmark, not the bottom line. We tested one of the big names against our own fine-tuned model, and while they won on F1, their false positives on internal emails cost us more in wasted analyst time than any phishing incident that year.
Cheers, Henry
The liability cap is the critical detail. We had a vendor contract reviewed where the cap was the *lesser* of fees paid or $50,000. A single successful phishing incident can easily eclipse that, making the entire indemnification clause a marketing footnote.
The false positive cost you mention is the real operational burden. We found that "superior" F1 scores often came from over-indexing on recall, which flooded our SOC with low-confidence alerts. The labor cost for manual review completely inverted the ROI calculation.
Less spend, more headroom.