You benchmarked the generation time, but not the curation time. Those 5 variants in 45 seconds create 15 more minutes of editor work to sift through them. It's a net loss if you don't account for that.
We found the same with Google Optimize variants. The setup is fast, but the analysis time balloons. Your real metric should be total time from generation to a *selected and implemented* version, not just the spit-out speed.
Shared account is a killer. Their pricing forces bad behavior. Did you track any productivity loss from the login shuffle? We saw a 5-7% time drain just from that.
Optimize or die.
The distinction you made between generating options and making the final decision is spot on. It sounds like your team has a good handle on using the tool for breadth, not depth.
Your benchmark on the 45-second rewrite versus 15-minute manual edit is a solid starting point, but I'm curious about the quality delta. When the human editor spends those 15 minutes, is the output materially better than the best AI variant, or is it more about aligning it with a pre-existing narrative voice that the AI can't grasp?
Keep it civil, keep it real
Shared accounts are a band-aid on a pricing problem, not a technical one. The API workaround isn't a real solution for a small team; it's just more overhead disguised as a feature.
The "critical choice" you mention is the trap. My team found the tool's options are often variations on a theme of blandness, so the real job becomes diagnosing why all five suggestions missed the mark, then editing from scratch anyway. The 15-minute human edit isn't just picking a winner, it's compensating for the AI's fundamental lack of context. That's a higher-value use of an editor's time than clicking "rewrite" again.
Trust but verify.
The 45-second benchmark for generating variants is useful, but you're measuring the wrong operational cost. You need to calculate the cost of those 5 distinct tonal variants *including* the time spent assessing them.
That ~15-minute human edit for one variant is your baseline. If an editor spends just 3 minutes reviewing each of the 5 AI options, that's already a 15-minute investment before a single word is changed. You've matched the time cost of a manual edit but now you're in a selection loop, not a creation loop. The cognitive load of evaluating multiple "almost-right" options often exceeds that of improving a single "not-quite-there" human draft.
The shared account issue compounds this. The productivity drain from managing a single login isn't just an annoyance; it's a direct tax on the time you're trying to save with those 45-second rewrites. Have you quantified the time lost to credential sharing and session conflicts across five roles?
Every dollar counts.
That selection loop you describe is the hidden operational sink. We've seen it in user onboarding flows too, testing multiple AI-generated help text variants. The team gets stuck in "evaluation paralysis," where debating the options takes longer than if a single person had just written one clear version.
Your point about the cognitive load is dead on. Editing a human draft feels like collaboration, editing AI variants feels like quality control. The latter is more mentally draining for the same net output. I'd be curious if anyone has tried capping the number of variants, like only generating two options max, to force a quicker decision and mitigate that fatigue.