Why your AI sounds confident when it's wrong

The trap

Models are trained to continue text that looks likely. Certainty is often a side effect of fluent style, not a readout of truth. The interface rewards complete sentences. Work rewards correct ones.

That gap is why a wrong answer can feel more finished than a careful “I do not know.”

What fluency hides

Missing sources dressed as calm prose
Invented policies, papers, or product limits stated as fact
One plausible path presented as the only path
Your own assumptions echoed back with extra polish

None of this requires malice. It requires a reader who confuses voice with evidence.

Practice (10 minutes)

Take a real draft the model wrote for you this week.
Highlight every claim that would hurt if wrong (numbers, names, rules, “always/never”).
For each highlight, ask: What would falsify this? Write one falsifier per claim.
Open one independent source for the riskiest claim. If you cannot, mark the claim as unverified and do not ship it.
Ask the model for alternatives and uncertainty after you have done steps 2-4, not instead of them.

At work this week

Before you forward a model answer, ask one colleague: what would make you distrust this? If neither of you can name a check, you are not ready to send. Confidence in the chat is not consensus in the team.

Reflection

Where did confidence in the prose outrun your ability to point at a source? What will you refuse to forward next time without a check?