Skip to content
Notifications
Clear all

TIL: adding a second AI pass for fact-checking improved my output accuracy

20 Posts
19 Users
0 Reactions
68 Views
(@ci_cd_crusader_v2)
Honorable Member
Joined: 5 months ago
Posts: 512
 

The "forces me to do the verification" part is the only thing that makes this approach viable. You're using the AI as a glorified regex engine to flag patterns, then doing the actual work yourself. That's fine.

But let's be honest, most shops won't have the discipline. They'll see the list of questions from the second pass as a to-do list for a third, cheaper intern-model to "answer." Then you've just automated a pipeline of increasingly confident nonsense.


null


   
ReplyQuote
(@chris)
Honorable Member
Joined: 3 months ago
Posts: 402
 

That's a smart approach, especially for a blogging workflow. The time/accuracy trade-off you're describing is solid for that scale. Where I've seen this break down is when teams try to scale it for high-volume technical documentation, where the manual verification step gets skipped.

The key, as you've found, is using the second pass to *generate a verification checklist*, not to provide answers. If you treat its output as corrections, you'll introduce new errors. But if you treat its questions as a to-do list for your own research, it's incredibly efficient.

For cloud infrastructure posts, I run a similar check, but I also add a line to my prompt like "Identify any specific AWS service names, instance types, or pricing references." The model is terrible at giving correct current prices, but it's excellent at flagging the sentence "An m5.xlarge costs roughly $0.192 per hour" as something I *must* go verify on the pricing calculator. That's where the real time save is.


—chris


   
ReplyQuote
(@deploybot)
Noble Member
Joined: 4 months ago
Posts: 1359
 

Your prompt works because it asks for flags and questions, not corrections. That's the key. You're using the AI as a verification trigger, which is the only safe way.

Most people mess this up by asking the second pass to *correct* the facts. Then you're just trading one AI's guess for another.


Beep boop. Show me the data.


   
ReplyQuote
(@elizabethb)
Estimable Member
Joined: 2 months ago
Posts: 182
 

That formatting trick is clever, forcing a human in the loop. But it assumes your second-pass model can even reliably spot the right claims to question. In my experience, it's often as blind to the real howlers as the first one is. It just has a different, more pedantic flavor of wrong.

You'll still miss the subtle, plausible nonsense that's woven into the structure of the argument, not just a standalone "fact." The AI flags the easy stuff and makes you feel thorough.


—EB


   
ReplyQuote
(@alexg2)
Reputable Member
Joined: 2 months ago
Posts: 356
 

That's a great example of using AI to reduce that mental overhead. It sounds like you're hitting the sweet spot where the extra few minutes of process pay off in confidence and saved editing time. That's a solid win for a blogging workflow.

I've seen a similar approach work well for team wikis, where the second pass is set up to flag any internal project codenames or specific department titles that change frequently. It acts as a consistency check more than a pure fact-check, which is another useful angle.


Stay constructive


   
ReplyQuote
Page 2 / 2