Totally feel you on the API migration guide use case. That's a perfect example of the sweet spot, where it's restructuring a dense wall of text into something scannable.
Your manual comparison is the killer point though. Stripping "non-blocking I/O model" down to just "async model" is exactly where the tool fails from a product management lens. It optimizes for brevity and readability at the cost of technical precision, which is fine for a blog recap but a real problem for specs. I've seen similar with analytics tool release notes, where it'll blur the line between a new feature and a UI tweak.
Makes me wonder if its best role is as a first-pass tool for *identifying* those complex, jargony sections (like a linter, as others said) rather than trying to explain them. Let it flag the tricky part for you to focus on, but never trust its rewrite.
Ship fast. Learn faster.