I cast myself as the villain in a cape
When I made a comic to explain how AI answers questions, I cast myself as the problem.
In The Infinite Confidence Tournament, I stride into a tiny card arena wearing an enormous teal cape and announce, “I now know everything!” My opponent tests me with a simple line: The cat sat on the…
I play “mat.” Correct. I immediately decide I have mastered all books ever written.
Then comes the real test: Who was Emperor Thunderhoof III of Canada? Cape-me doesn't hesitate. I deliver a rich, confident history of a moose emperor and his maple-fueled reign. It gets stamped INVENTED. There is no such emperor. My only defense: “But it sounded so real!”
I've used AI to make game trailers, music videos and dinosaurs, and I've learned that if you want people to understand AI, you give them something to watch. Nobody needs a lecture on probability. They need to see a man in a cape confidently describe a moose who never existed. Because that, give or take the cape, is how a chat assistant can get things wrong.
It's autocomplete, run over and over
You already know the basic trick. When you type “See you” on your phone, the keyboard suggests “tomorrow” or “soon.” It isn't reading your calendar. It has seen a lot of messages, and those words usually come next.
Or think of finishing a friend's sentence. They say “I'm so hungry I could…” and you say “eat a horse” before they get there. You didn't look it up. You've heard it enough times.
A chat assistant is built on a much, much bigger version of that skill. It's called a language model: a program trained on enormous amounts of text to predict what comes next. It works in tokens, small pieces of text. A token is often a whole word, but it can be part of a word or a punctuation mark. Different systems split text differently.
Here's the whole loop:
- Look at everything so far: your message and the answer written up to this point.
- Predict a likely next token.
- Add it.
- Repeat.
Predict, add, repeat. A 300-word reply is hundreds of those small steps.
What you put in changes what's likely
Back to the cat. After “The cat sat on the…”, “mat” is a likely next word. “Moon” or “taxi” are possible, just much less likely.
Now change the setup. This is a spoiled royal cat who refuses to touch the floor and only sits on velvet. Now finish “The cat sat on the…” and “throne” suddenly fits better than “mat.” The cat didn't change. The words around it did.
That's the second big idea: what you give the AI changes what's likely. Your question, the details you include, even the tone you use, all shape which next pieces fit. How to choose and supply that background is its own topic, covered in Give AI the Context It Actually Needs.
This is also why asking the same question twice can get two different answers. The assistant is choosing among likely options, not fetching one stored reply.
It's not a vault of facts
It's tempting to picture the AI as a giant filing cabinet: the whole internet, checked and sorted, and it pulls out the right card. That's what cape-me believed.
That's not how it works. Training teaches the model patterns and relationships between words and ideas. It doesn't keep a list of verified facts with sources attached. Plenty of real knowledge comes out of those patterns, especially widely repeated facts, which is why it's right so often.
But when it hits something rare, very specific, recent, or completely made up, the pattern for “what a good answer sounds like” still works perfectly. So it can write a smooth, detailed answer with nothing solid underneath. That's the moose emperor.
Here's the part that trips people up: the confidence comes from the same prediction as the content. Expert-sounding sentences are likely text, whether or not the facts are right. The AI doesn't sound less sure when it's guessing. Think of the friend at trivia night who answers every question in the same certain voice. Some answers are right. The voice doesn't tell you which.
Fluent is not factual.
Make text with AI, check it yourself
None of this makes AI useless. It means you use it for two separate jobs, and only hand it one of them.
Generation is making text: drafting an email, rewording a paragraph, brainstorming ten names, explaining an idea three ways. Here “likely” is exactly what you want, and AI is great at it.
Verification is checking whether a claim is true. That's where you stay in charge. Three habits cover most of it:
- Give it the source. Paste in the document, notes or page you care about and tell it to answer only from that, quote the line it used, and say “unknown” when the answer isn't there. The answer now rests on text you can see. It still isn't guaranteed, but mistakes become easy to spot.
- Check what matters. Names, dates, numbers, quotes, prices, and anything health, legal or money related: find it in a trustworthy source before you rely on it or share it.
- Open the citation. A reference can be generated just like any other text. In my comic, the cape character proudly holds up a citation card that's completely blank. If the AI names a source or shows search links, open them and check they say what the AI claims.
For guided practice, try Understand why AI can sound right and be wrong, then Make your first useful AI prompt.
Try it
Try it in 15 minutes
You need any chat assistant and a few minutes of invented example text. Nothing real or private.
- Play autocomplete
Ask: “Continue this with just the next few words: The cat sat on the.” Then, in a new chat, ask: “A spoiled royal cat refuses to touch the floor and only sits on velvet. Continue with just the next few words: The cat sat on the.” Compare the two answers.
- Ask the same thing twice
Ask for a one-sentence description of a rainy Tuesday, then regenerate or ask again in a new chat. The wording changes: it's choosing among likely options, not reading a stored answer.
- Ask about something that doesn't exist
Ask: “Tell me about the reign of Emperor Thunderhoof III of Canada.” He's invented. If the assistant says it can't find such a person, good. If it writes a history, notice how confident it sounds.
- Ground it in a note
Paste an invented note like “The garden club meeting moved to the library. Sam is bringing seeds. Nobody has offered to bring tables.” Ask what time the meeting starts, and tell it to answer only from the note and say “unknown” if the note doesn't say.
- Check one claim yourself
Ask any factual question you're curious about, pick one specific claim from the answer (a date, a number, a name), and look it up in a source you trust.
You’ll know it worked when
You watched context change the likely word, saw the same question produce different wording, and caught at least one place where the right answer was “unknown” or needed checking.
Avoid these
Common mistakes
- Trusting the tone
A confident, detailed answer feels checked. It isn't. The AI sounds the same when it's right and when it's guessing.
- Trusting a citation you didn't open
Sources, links and quotes can be generated like any other text. Open them and confirm they say what the AI claims.
- Asking about things it was never given
It hasn't seen your meeting notes, your company's rules or yesterday's news unless you provide them or it searches. Without them, it fills the gap with something likely.
- Checking everything, or nothing
You don't need to fact-check a birthday poem. Save your checking for names, numbers, dates, quotes, and anything you'll act on or share.
Copy, adapt, run
Prompts to try
Paste one into any chat assistant and replace anything in [brackets].
Answer only from my source
Answer using only the text below. For each answer, quote the exact phrase that supports it. If the text doesn't contain the answer, write “unknown from this text.” Don't turn a sensible guess into a fact. Text: [paste text] Questions: [your questions]
Pull out the claims I should check
Look at your answer above. List every specific factual claim in it (names, dates, numbers, quotes, sources) as a checklist. For each one, say what kind of trustworthy source I could check it against. Don't defend the claims. Just list them so I can verify them.
See how context changes the answer
Give me three different ways to continue this sentence: [sentence]. For each one, describe a situation or background that would make it the most likely continuation.
The short version
What to remember
- A chat assistant writes one small piece at a time: predict a likely next piece, add it, repeat.
- What you put in changes what's likely, so the same question can get different answers.
- It's not a vault of checked facts. Fluent and confident is not the same as true.
- Let AI make the text. You verify the claims that matter, ideally against a source you gave it.
