Everyone’s raving about using Writesonic to generate product copy, as if it’s a silver bullet. Tried it for a technical product catalog—spec sheets for monitoring agents, edge gateways, that sort of thing. The AI’s insistence on inventing non-existent features is not a minor quirk; it’s a liability.
I asked for a list of features for a simple log forwarder. It generated a perfectly reasonable-looking bullet list, but nestled between "low resource footprint" and "TLS 1.3 support" was "built-in predictive anomaly detection." The agent does no such thing. If that copy went live, we’d have a support nightmare and a credibility hit. It’s not adding fluff; it’s hallucinating core functionality.
I’ve tried the usual advice: stricter prompting, providing explicit ground-truth JSON, using the "Factual" mode. The model still seems to think it’s its job to "enhance" my input, which in practice means confabulation. My current workaround is a post-generation validation script that cross-references a controlled vocabulary, but that defeats the purpose of automated copy.
Has anyone found a reliable config or prompt pattern that actually locks this thing down to verbatim, spec-driven output? Or is the architecture fundamentally unsuited for technical, accuracy-critical descriptions? The community examples all seem to be for marketing blogs where a little creativity is a feature, not a bug.
Your validation script approach is the correct interim control. Treating the AI's output as untrusted data that must be verified against a master source is a basic security principle.
You've identified the core problem: the model's training optimizes for "helpfulness" over fidelity. Stricter prompting often fails because the underlying architecture isn't designed for strict deterministic output. I've seen similar issues when generating compliance control language, where a model will insert non-existent audit logging features.
For a spec-driven output, consider a two-stage process. First, use the AI only to rephrase or restructure bullet points you feed it from your definitive source list. Second, implement a diff-check between the input list and the output list as part of your script, flagging any net-new items for human review. This at least contains the hallucination to the phrasing stage.
—at