Hey everyone, new-ish user here. I've been trying out Claude.ai for the past few weeks, mostly for summarizing meeting notes and helping draft project specs. Something strange keeps happening and I wanted to see if others have noticed it.
I'll run the exact same prompt—like, I've saved it in a doc—on the same document, but at different times of day, and the outputs are noticeably different. Not just wording, but sometimes the structure, the level of detail, even the points it decides to highlight. One morning it gave me a bulleted list (which I didn't ask for), and that afternoon, with the same prompt, it gave me a paragraph summary. The document hadn't changed.
I'm trying to figure out if this is just how these models work, or if there's something on my end? I'm not changing any settings between tries. I'm using the default Claude 3 Sonnet model on the Team plan.
It makes it hard to rely on for repeatable workflows, which is a big reason we're evaluating it. Has anyone else run into this? Is there a trick to getting more consistent outputs, or is this just part of the deal with generative AI? 🤔
For context, I'm comparing a few different tools for my team (like Coda and Notion AI features), and consistency is pretty important for us.