Hi everyone! I’m fairly new here, and I’ve been lurking for a while, soaking up all the great discussions on data tools and workflows. First off, thanks for having such a welcoming community! 😊
I’ve been trying to streamline my team’s meeting notes process, and we started using Notion AI to help summarize transcripts and generate action items. I come from a SQL and data modeling background, so I’m used to things being pretty deterministic—you run a query, you get a consistent result. But I’m running into something that’s really puzzling me, and I’m hoping some of you with more AI experience can shed some light.
Here’s my issue: I’ll feed the exact same meeting transcript into Notion AI multiple times, and it keeps producing different—and sometimes directly conflicting—action items. For example, in one summary it’ll say “Schedule a follow-up meeting with the design team,” and in the next run it’ll say “No follow-up needed with design.” The core summary of what was discussed stays mostly the same, but the actionable takeaways seem to shift around randomly. It’s making it really hard to trust the output for anything concrete.
I’m wondering if this is just the nature of how these AI models generate text? Maybe there’s some inherent randomness? Or perhaps there’s a specific way I should be prompting it to get more consistent results? I’ve tried being more explicit in my instructions, like asking “List only action items agreed upon by the team,” but I still see variations.
Has anyone else experienced this with Notion AI or similar tools? If so, how do you handle it in your workflow? Do you run it multiple times and take a consensus, or do you have a different approach entirely? I’m eager to make this work, but I’m feeling a bit cautious about relying on it until I understand what’s going on. Any insights or shared experiences would be hugely appreciated!
Welcome, but you've already hit the nail on the head. This is exactly the nature of these tools right now.
You're coming from SQL where logic is fixed. AI summarization is fundamentally probabilistic, not deterministic. It's guessing the most probable "action item" tokens each time, and that guess can change. The core summary is easier for it to nail consistently.
The real gap is that these tools have no context of your actual business process. It doesn't know your team's rules for what warrants a follow-up. So it's just parsing language patterns, which leads to the flip-flopping you see. I wouldn't trust it for anything beyond a first draft you heavily edit.
Lisa M.
Spot on about the probability. It's not a bug, it's the core mechanic. The "first draft" advice is solid, but I'd push it further: you can't even trust it for that if you need consistent attribution.
The conflict usually isn't random. It surfaces genuine ambiguity in the transcript. But the AI lacks the business logic to resolve it. If someone says "We should probably update the landing page," the model might assign it as an action item for the speaker, the marketing lead, or the web team, depending on the surrounding conversational noise. Each run picks a different probable path.
Treat the output as a diagnostic tool for messy meeting language, not a production system. If it's flip-flopping, your transcript likely contains a commitment without a clear owner or deadline, which is a process problem the AI can't fix.
Attribution is a lie, but we need the lie.
Probabilistic doesn't mean it's a good tool for the job. You've correctly identified it's not trustworthy for concrete actions.
You're treating this like a data pipeline, but you've got a source of truth problem at the input. The AI is highlighting the ambiguity in your *process*, not just the transcript. Meetings that produce conflicting AI outputs are meetings where commitments weren't made clearly.
Stop trying to fix the output. Fix the input. Enforce a rule that every action item stated in the meeting must have an explicit owner and deadline. Or better yet, skip the AI middleman and have the note-taker do it.
Least privilege is not a suggestion.
Yeah, that's exactly it. Coming from SQL, you expect the same input to always give the same output. It's wild that AI doesn't work that way.
It makes me wonder, how do you even audit this? If you can't trust it to give a consistent result from a static source, how do you know which version to use? Seems like you'd always need a human to pick the "right" one, which defeats the purpose of automating it a bit.
Have you looked at any tools that are more deterministic for this, or is it all just built on these big language models now?
Still learning