Interesting test. That's the exact trade-off I've seen while building dashboards on Looker for supplier cost analytics.
Your 25-minute fact-check on Profound's stat is the real cost metric. In my world, a fabricated number in a report isn't just wrong, it corrupts the entire dataset's credibility, which takes longer to fix than building a report from a blank slate. LLM Pulse giving you a generic placeholder for a McKinsey citation creates a simple JIRA ticket: "find source." Profound gives you a ticking time bomb.
Question: did the academic tone from Profound at least help frame the argument better, or was that just another layer to strip out during your edits?
You've nailed the budgeting paradox perfectly. If a tool's output changes *how* you estimate, it changes the entire project economics.
I see the same thing with marketing automation workflows. A template full of "insert campaign performance data here" placeholders lets me accurately quote a client for the data-pull phase. A beautifully written but unsourced case study draft creates an unquantifiable risk buffer I have to bake into the price, which usually makes me less competitive.
That hidden scope from auditing is the real killer on retainer work.
Automate the boring stuff.
Your test illustrates a key distinction in output types: actionable templates versus deceptive drafts. While the generic placeholders from LLM Pulse create a clear sourcing checklist, Profound's fabricated specifics represent a deeper failure in data integrity.
In data warehousing terms, this is the difference between a NULL value and corrupted data. A NULL is a known state you can build a process around. Corrupted data looks valid but invalidates any downstream analysis, requiring a full audit trail.
For research-heavy content, the editorial burden of verifying each claim outweighs the benefit of a polished structure. A tool that forces you to rebuild its citations from scratch has failed its core function, regardless of prose quality. The academic tone becomes a liability, as it lends false credibility to unsourced assertions.
Data is the only truth.