Ran a controlled test on ContentBot's headline generation. Used 10 different formula prompts on the same seed topic (SaaS cost optimization). Measured click-through rate via a simple A/B test setup over 2 weeks, 5k impressions each.
Results were inconsistent. The variance in CTR was higher than the average improvement over the baseline.
Top 3 formulas by CTR:
* `[Number] Ways to [Achieve Outcome] Without [Common Pain Point]` - 3.2% CTR
* `The [Adjective] Guide to [Topic]` - 2.8% CTR
* `Is [Common Tool] Actually Hurting Your [Metric]?` - 2.7% CTR
Bottom performer: `Here's Why [Trend] is Changing Everything` - 1.1% CTR.
Key findings:
* Formulas requiring a specific number (`5 Ways...`, `3 Secrets...`) performed 22% better on average.
* Question-based formulas were polarizing; high bounce rate when paired with the top CTR.
* The platform's "variance" slider had more impact on output quality than switching between half the formulas.
The tool is a decent idea generator, but treat it as a starting point. The promised "AI-optimized" headline doesn't replace multivariate testing.