I just wrapped up a head-to-head test on this exact use case. Needed to generate API docs for a new feature. Here's the prompt I used for both:
**Prompt:** "Write a clear, step-by-step guide for developers to authenticate via OAuth 2.0 with our new API. Include prerequisites, error code handling for invalid tokens, and a short cURL example."
**Profound's Output:** Was very structured and formal. The steps were logically ordered, and the cURL example was technically correct. However, it buried the crucial note about token expiration in the troubleshooting section. The tone felt a bit robotic for developer docs.
**Searchable's Output:** Felt more conversational right away. It led with the prerequisites in a handy "Before you start" box, which I liked. The cURL example included inline comments explaining each flag. But, it completely omitted the specific error codes I asked for.
**My editing notes:**
- With Profound, I had to move key warnings higher up and rewrite sentences for clarity. The bones were good, but it needed a voice.
- With Searchable, I had to manually add the error handling section from my own knowledge. The readability was better out of the gate, though.
Verdict? Profound if you need a solid, factual draft you're willing to heavily edit for tone. Searchable if you want a more engaging first draft but might need to fact-check for technical completeness. For strict technical docs, I'm leaning towards Profound as a starting point, but it's close.
Anyone else run them through similar tests? Curious if you had different results.
—b
—b
So you got a usable skeleton from Profound but had to rewrite the tone, and Searchable gave you better readability but missed a core requirement. Sounds like both still need significant human editing, just in different areas.
For API docs, missing error codes is a deal-breaker. A developer will hit that first.
Did you run the cURL examples? Sometimes the "helpful" inline comments get the syntax wrong.
Benchmarks or bust.
Missing error codes in Searchable's output isn't just a missed requirement, it's a liability. Your editing notes show both tools failed, just in opposite ways.
You're now the human quality gate, patching holes in AI-generated scaffolding. How many of these "quick wins" does it take before you've spent more time fixing than writing from a blank page?
Your mileage will vary