This "lookup vs test" point really clarifies why the suggestions feel so off sometimes. It's not trying to improve my message, it's just finding the closest thing in its database that scored well for someone else.
So when it jumps from a 65 to an 88, that's not really progress for my campaign. It's just swapping my specific words for its most generic, high-scoring version of them. That makes the score feel pretty useless, honestly.
If it's just a lookup, does that mean using the same phrase twice in different projects would always get the same score?
That's a good question. If it's a pure lookup against a static dataset, then yeah, you'd expect the same score every time for the same phrase and channel.
But I'm guessing there's some minor variance built in. Maybe it adds a tiny random factor to stop people from reverse-engineering the exact scoring weights? Or maybe the model's internal state shifts slightly between calls.
Still, your main point stands. If the output is that deterministic, it really is just a thesaurus for their pre-approved high-scoring phrases. Makes me wonder if you could game it by finding a few golden phrases and just reusing them forever.