My results after 30 days: Did it actually improve CTR?

Alright, I’ve been wanting to write this up for a while. As someone who lives in our BI tools and A/B tests dashboard copy, email subject lines, and CTA buttons constantly, I was skeptical about Anyword. The promise of “AI that predicts performance” felt a bit like marketing fluff. So, I committed to a 30-day, methodical trial across three of our main channels: LinkedIn sponsored content, Google Search ads, and our weekly newsletter CTAs.

My goal was simple: measure any uplift in CTR against our historical, human-written benchmarks. I used our standard Tableau performance dashboards to track this, segmenting by channel and content type. Here’s the setup I used for a fair comparison:

* **Control Group:** Our best-performing, human-written copy from the last quarter.
* **Test Group:** Anyword-generated variations (I used the “Performance” score heavily).
* **Methodology:** For each campaign/email, I ran an A/B test (50/50 split) for a minimum of 72 hours or until statistical significance was reached (using a 95% confidence level).

Here’s a summary of my aggregated results:

| Channel | Sample Size (Impressions/Sends) | Avg. CTR Control | Avg. CTR Anyword | Lift | My Take |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **LinkedIn Ads** | ~250k impressions | 1.42% | 1.58% | **+11.3%** | Consistent, modest improvement. It excelled at refining value propositions. |
| **Google Search** | ~85k impressions | 3.87% | 4.21% | **+8.8%** | Good for expanding keyword-focused variants, but our best human copy still won sometimes. |
| **Newsletter CTAs** | ~45k subscribers | 2.15% | 2.14% | **-0.5%** | Essentially no lift. For our engaged audience, the "predictive" tone sometimes felt generic. |

**Key Workflow Observations & Pitfalls:**

* **It’s a fantastic brainstorming tool.** Even when its top-scoring variant didn’t win, the list of suggestions often sparked a better idea for our human writers. It shaved hours off our ideation phase.
* **The "Performance" score is directionally useful, but not gospel.** I learned not to blindly trust the 90+ scores. The “Brand Fit” slider was crucial—setting it higher yielded more on-brand, usable suggestions.
* **Data governance folks, take note:** You’ll want a process for what gets exported and logged. I set up a simple log in Airtable to track which AI variant was used in which test for auditability.
* **Major Pitfall:** It can get repetitive. If you generate lots of copy for the same product, you’ll start seeing very similar phrasings. You *must* provide fresh, unique input briefs.

**So, did it actually improve CTR?** Yes, for direct-response, ad-focused copy where the goal is a clear, immediate click. The lift was statistically significant and operationally worthwhile for those channels. For nuanced, relationship-based communications (like our newsletter), the impact was negligible.

Ultimately, I see it as a powerful addition to our self-serve analytics stack. It won't replace a skilled copywriter who knows our audience deeply, but it acts like a incredibly fast, data-informed junior analyst for your copy team. I’ve recommended we keep the license, but restrict its use to specific use cases (paid social, meta descriptions, PPC) where we've proven the lift.

Would love to hear if others have run similar structured tests, especially in B2B contexts. What was your experience with the performance scoring versus real-world results?

~jenny