My workflow is technically sound but the output is generic. I suspect the problem is in the prompt chain and lack of human-in-the-loop stages. Metrics are good (low toxicity, high readability) but engagement is down.
Current setup:
1. Keyword cluster -> GPT-4 for outline
2. Outline -> fine-tuned model for draft
3. Draft -> grammar/spell check
4. Publish
The issue is step 2. The fine-tuned model is trained on our own past content, which amplifies existing blandness.
Proposed fixes I'm testing:
* Inject a "contrarian" prompt stage between 1 and 2. Force the model to argue against the outline or suggest an unexpected angle.
* Replace the generic grammar check with a style editor step using a model tuned on high-engagement examples (not our own corpus).
* Add a mandatory human "edge pass" where the editor's only job is to insert one surprising statistic, counter-intuitive claim, or personal anecdote.
Has anyone structured a similar corrective stage into their pipeline? What's your success rate on originality scores post-edit?
Prove it with a benchmark.
You've diagnosed the core issue perfectly. Training on your own corpus is a textbook case of a feedback loop that just reinforces your existing voice, or lack of it.
The "contrarian" prompt stage is a clever idea, but I'd caution that it can sometimes produce forced or gimmicky output. The human "edge pass" is likely your most reliable fix. We've seen similar setups where that final human step isn't for major edits, but solely for injecting a specific, tangible detail it makes all the difference. The originality score jumps because you've added something the model couldn't pull from the training data.
Have you considered swapping step 2 entirely? Use a general model for the draft, but with a prompt heavily conditioned on a style guide from an external, high-engagement source?
Keep it constructive.