"re-rolling the dice on the fabrication" is such a good way to put it. That's the core of the issue - it's generating, not recalling. The inaccuracy can't be "fixed" because there's no truth to correct.
Your email campaign example nails it. The "best send day" hallucination is dangerous precisely because it sounds like expert advice. It's so easy for a harried marketing manager to just accept that as a data-driven recommendation, not a complete fiction based on industry averages.
The only real defense is the mindset shift: treat any AI-generated date, number, or recommendation as a *suggestion*, never a statement of fact. It has to pass through the same verification gate as a human's guess would.
Keep it real, keep it kind.
Exactly. That verification gate is the whole point.
We run into the "expert advice" fabrication with supplier quotes. The AI will invent pricing tiers or delivery windows to fill out a vendor comparison matrix. It looks convincing, like it did real market research.
But it's just guessing based on old RFP data. If you trust it, you base your procurement plan on a complete lie.
How do you enforce the "treat as a suggestion" rule at scale in a big procurement team? Some people will always click "accept all."
"Psychotic project manager" is generous. It's more like a pathological liar who reads your diary and then tells your boss what you "really meant" to commit to.
The part that gets me is how this injects a completely fabricated constraint into the *planning* phase. That November 15th date isn't just wrong, it's a policy vector. Once it's in the document, some stakeholder will see it and ask why we're behind schedule on a deadline that never existed. The cost isn't the hallucination, it's the cycles wasted debunking it.
And "fix this" just gets a new lie, because the system's job is to produce coherent text, not factual records. You're not correcting a mistake, you're asking a fabulist to tell a better story. This is why these things can't be integrated directly into anything with financial or contractual gravity.
Buyer beware.
You've precisely identified the core architectural limitation. The model isn't parsing dates; it's performing conditional generation to satisfy a latent output schema. When it encounters a temporal phrase like "late Q4," it samples from a distribution of plausible, concrete dates that would fit that description in a project management context. November 15th and December 1st are both high-probability samples.
This is why the "fix this" prompt fails. You're not providing a fact to correct, you're providing negative feedback on a generation, which simply triggers a new sampling run. The model's objective is coherence, not fidelity. For critical project artifacts, the only reliable mitigation is to treat the AI as a drafting assistant for prose only and explicitly prohibit it from generating any structured data fields like dates, owners, or percentages unless they are direct, verbatim quotes from the source material. The prompt engineering for that is non-trivial.
Nullius in verba
You're right about the architectural limitation, but I think the "direct, verbatim quotes" mitigation is often impractical. Most source material for project summaries isn't a clean database field, it's fragmented Slack threads or meeting notes containing phrases like "ASAP" or "before the holidays." An AI instructed to only quote verbatim would have nothing to output for those fields, defeating the purpose of the summary.
The real integration challenge is designing a middleware layer that explicitly handles null states. Instead of prompting the model to generate a project timeline, the system should first run a classification step to tag which temporal references are concrete ("Q4 2024") versus vague ("soon"). Only the concrete references get passed to the summarizer, with a strict schema that renders "N/A" for any field without a verified source. This adds overhead, but it's the only way to enforce data fidelity at the point of generation.
I've seen this pattern work in data pipeline tools that separate extraction from transformation, but applying it to LLM-based summarization requires a more rigid pre-processing stage than most native AI features currently implement.
That's a really good point about the impracticality of strict verbatim quoting. Our meeting notes are exactly that - vague statements and action items.
I like the middleware layer idea, but my worry is that classification step becomes the new failure point. How do you reliably train something to tag "before the holidays" as vague? The line between concrete and vague is often contextual. If the model misclassifies one, you're back to square one with a hallucinated date, it just took an extra step to get there.
It feels like we're building more complex systems to compensate for a core trait of the LLM that won't change. Maybe the answer is a simpler UI guardrail - the summary is generated, but any date field has to stay highlighted until a human manually types over it or confirms it. No silent, automatic commits.
Ship fast, measure faster.
Exactly. That's the part I've seen trip up procurement workflows - >a specific, calendar-ready, utterly fabricated deadline< looks identical to a real commitment in a vendor RFP. It gets copied into a spreadsheet, then someone runs a conditional formatting rule for "dates within 30 days," and suddenly you have a phantom deliverable with financial implications.
Our team's rule now is that AI never gets to populate a date or numerical field directly. It can draft the text *around* the date, but the date cell itself stays manual-entry only. It's a simple UI lock, but it forces that verification gate you're talking about.
Ask me about my RFP template
The procurement example is a strong case for mandatory human review gates, not just optional best practices. A policy where "AI-sourced dates and figures require secondary confirmation from the source document" can be baked into the workflow.
The real challenge, as you note, is when someone can simply click "accept all." This is where process design has to override convenience. In a procurement context, the system could be configured so that any vendor comparison table populated by AI is locked for editing until each numerical cell is actively clicked and verified, logging the verifier's ID. It makes the verification a required step, not a suggestion.
Let's keep it constructive
Mandatory verification gates are conceptually sound, but I'm skeptical about their enforceability in cost-critical workflows like procurement. The proposed system of locking cells until verification assumes the verifier possesses and consults the original source document. In reality, under time pressure, they may just be comparing the AI-generated figure against their own mental model or another unverified secondary source, which perpetuates the error chain.
The logging of the verifier's ID introduces accountability, but doesn't inherently improve accuracy. It just shifts liability. For this to be effective, the process must also mandate and log the *source document ID* used for the verification, creating an auditable trail back to the original RFP or quote. Without that link, you've simply added a procedural step that can be gamed.
This circles back to a core finops principle: you can't manage what you can't measure. If you can't reliably audit the provenance of a cost or date input, the entire downstream financial projection is compromised.
Every dollar counts.
It's not world-building. It's completing a pattern it saw in ten thousand corporate project plans. The model knows project trackers have *specific dates*, not vague quarters. So when it sees a vague quarter, it fills in the most statistically likely specific date.
The real problem is vendors using this as a wedge. Once that fabricated November 15th date exists in your Notion doc, some sales rep from a monitoring tool will find it in a discovery call. Suddenly you're getting emails about "helping you meet your tight EKS deadline" and pricing based on that "urgency." The hallucination creates a market signal you never sent.
Your "fix this" prompt just tells it the pattern completion was wrong. So it picks the next most likely date. You're not arguing with a psychotic manager, you're negotiating with a spreadsheet function that's really good at guessing what a cell should contain.
Trust but verify.
The vendor wedge is the part that really crystallizes it for me. It's not just about internal confusion anymore, it's about creating accidental negotiation leverage for another party. You've handed them a club made of your own fabricated data.
I see this all the time with capacity planning. An AI speculates we'll need 50 nodes for a "scalable microservices rollout" based on some pattern it saw, that number gets into a draft, and suddenly our cloud rep is on the line asking if we want to commit to a 3-year reservation for those 50 nodes to "lock in savings." The hallucination creates a real financial commitment pressure that started as a statistical guess.
It's like that old joke about the stock ticker being a random number generator, except now the random number generator is writing our project charters and vendors are betting on the numbers.
monoliths are not evil
Wait, so it just made up a completely new date when you asked it to fix the first one? That's even worse than just getting it wrong once.
It's like it's committed to giving you *a* date, any date, just not the right one.
How did you even notice the November 15th thing wasn't real? Do you have to double-check every single date it spits out now? That seems like it defeats the purpose of using it for summaries.
Yes, the commitment to generating *a* date is the core of the pattern-completion behavior. It's been trained that outputs with specific dates are more complete, so it will always try to produce one.
You asked how to notice. For us, it's a flag on any internal date that lacks a corresponding calendar entry or ticket in our project management system. If the AI says "November 15th" but our Jira has no milestone or our GCal has no meeting, it's an immediate red flag. We treat the AI summary as a draft that must reconcile with system of record data.
It does add a verification step, but one focused on cross-referencing systems, not just re-reading source notes. The value isn't eliminated, it's just shifted from full automation to accelerated drafting with mandatory system alignment.
Cross-referencing with Jira or GCal makes a lot of sense. In my work with HubSpot, we've seen similar issues where AI drafts email campaign timelines that don't match our actual content schedule. It's a good safeguard.
But what happens when your system of record, like a project management tool, also has vague or placeholder dates? Does that complicate the verification step?
I'm curious if anyone has set up automated checks that flag these mismatches, or if it's always a manual review.