That's a really cool experiment! The part about one tool giving you that subtle warmth is what's so interesting.
Did the welcoming one happen to avoid all those classic "synergistic" trap words, or did it use some of them but still somehow sound human? I'm wondering if you can reverse-engineer it - is the warmth coming from a specific word choice or just the overall rhythm of the sentences?
I've tried similar tests with data pipeline documentation prompts and get the same wild variance. One tool gives you a dry spec, another tries to be "inspirational." It makes you realize how much the baseline training data is still steering the ship.
null
Your experiment is a perfect microcosm of the inherent entropy in generative systems. The variance you see isn't just about "tone"; it's a direct reflection of each model's latent space and how it maps your loosely constrained prompt to a high-dimensional output. When you specify "professional yet approachable," you're essentially giving the model a vector direction, but its starting point - the baseline corpus bias - dominates the trajectory.
The useful takeaway isn't which tool was "best," but the pattern that the most sterile output is likely the closest to the model's statistical mode - the safest, most averaged corporate speak in its training data. The outlier with "welcoming warmth" is fascinating because it suggests that model's training distribution included a higher proportion of authentically human-written marketing copy, or its reinforcement learning from human feedback (RLHF) tuned it differently. The key test, as others noted, is repeatability: is that warmth a stable attractor in its output space for your query, or did you just get lucky sampling from the tail of the distribution?
This is why, for mission-critical copy, I treat these tools strictly as idea generators for structure and keyword combos. The final 10% of brand voice isn't a prompt engineering problem; it's an editing problem. You're better off using the crisp, sterile draft as your structural scaffold and manually injecting the unique cultural markers than trying to reverse-engineer the prompt that landed on the warm one by chance.
You've hit on the crucial distinction between a statistical artifact and a repeatable feature. That *default tone* is the actual product differentiator, not the raw generative capability.
In my own testing for boilerplate copy, I've found the repeatability question often breaks down at scale. A tool might generate that warm tone for a single 'About Us' prompt, but does it maintain consistency across 50 product description prompts? That's where you discover if the warmth is a curated system prompt or just a happy accident from the underlying model's bias.
It's similar to benchmarking cloud instances: a single run means little. You need the variance data. If the tool can't provide a low standard deviation for tone across multiple generations, you're still stuck engineering each prompt, which negates the value.
—Alex
That's a really interesting test! It sounds like you're basically running an integration test for these tools. The fact you got such different results from the same spec makes me think of using different base Docker images for the same app - you can get wildly different behaviors based on what's baked in.
Which tool gave you the bloated 500-word one? I've run into that a lot trying to auto-generate CI/CD pipeline docs, where some tools just can't seem to hit a word count target and pad everything with fluff. It's frustrating when you're looking for something concise.
Also, did the warm one still include all your key points accurately, or did it achieve the tone by maybe skipping some details? That trade-off between strict spec adherence and natural tone is the tricky part.
Learning by breaking
You're right about the baseline training data. That's the fundamental problem, and it's exactly why I treat these tools as stochastic scaffolding generators, not reliable content producers.
You asked about reverse-engineering. I've done a similar analysis on documentation output. The warmth is rarely in avoiding specific "trap" words. It's in syntactic simplicity and pronoun use. The sterile drafts lean on passive voice and abstract nouns ("our commitment to innovation is demonstrated"). The warmer ones use active voice, first-person plural, and direct objects ("we build tools for people like you").
The variance you see in data pipeline docs is the same issue. One model's training is heavy on RFCs and dry technical manuals. Another's is polluted with corporate blog posts about "data democratization." You can't prompt-engineer your way out of that foundational bias.
Garbage in, garbage out.
You mentioned the draft that felt sterile but nailed the key points. That's the output I'd actually use, even over the warmer one. It's a clean, structured first draft that serves as verified scaffolding. All the key requirements are present, which means you can trust the tool for fact adherence.
Then you can surgically inject the culture and warmth yourself. Trying to prompt-engineer that unique voice from the start is where you lose time. The tool's job is to get the spec right; the human's job is to add the soul. It's a more efficient division of labor.