It's wrong in patterns

The confident wrong answer

Seven issues in, and I've spent all of them on what this stuff is good for. Here's the other half.

AI gets things wrong often enough that you have to account for it. The mistakes themselves aren't really the issue. The issue is that a wrong answer looks exactly like a right one. Same tone, same structure, same confidence.

Think about how you normally catch bad information. A coach at a clinic talking past what he actually knows usually gives himself away. A weak study tells on itself in the limitations section. AI gives you its best guess in the same voice it uses for accurate information.

The useful part is that the mistakes follow patterns. There are a handful of specific ways it goes wrong, and once you know them, you can stop double checking everything and start looking twice at the four things that actually deserve it.

The four ways it goes wrong

• It invents sources. This is the big one for our field. Ask for research on a topic and you might get a citation with a real journal, a plausible title, authors who genuinely work in that area, and a publication year that fits. The paper doesn't exist. Or it exists and says something different than what you were told. The tool is producing what a citation on that subject should look like, which is a completely different task than going and finding one.
• It's confidently out of date. Every model has a cutoff date for what it learned, and it doesn't always know what changed afterward. Ask about a piece of technology, a company's product line, or a rule that's been updated, and you can get an answer that was accurate two years ago delivered like it's accurate today.
• It does arithmetic badly. Volume totals, percentage calculations, unit conversions, adding sets across a training week. These are the things it gets wrong most quietly, because the number it gives you usually lands close enough to look reasonable and far enough off to matter.
• It agrees with you too easily. Push back on something it told you and it will often fold, even when it had it right the first time. Ask a question that implies an answer and you'll usually get that answer back. This is the hardest one to catch, because agreement doesn't feel like a mistake. It feels like being right.

Inside the Lab: what to actually do about it

Verifying everything would cost you all the time this was supposed to save. The move is to verify the things that match those four patterns.

• Anything with a citation attached, open the link. Go to the actual paper, not the summary of it. If there's no link, ask for one, and if it can't produce one, treat the claim as unsourced until you find the source yourself.
• Anything time-sensitive, ask what it's basing that on. Product specs, requirements, current recommendations, anything that changes. A quick question about how current the information is will usually surface the uncertainty on its own.
• Anything with a number in it, spot check one line by hand. Same rule from the spreadsheet issue a few weeks back. Take one week, add it up yourself, see if the totals match.
• Anything you pushed back on, ask it to defend the original answer. If it switched positions, make it explain why. Sometimes you were right and it corrected properly. Sometimes it just gave you what you seemed to want.

Underneath all of this is one question worth asking before you act on anything: did it retrieve this from a source it can show me, or generate it because that's what the answer usually looks like?

What it looks like in the real world

You ask about research on a training method you're thinking about using. You get three studies back with a clean summary of each.

Two of them have links. You open both and they say roughly what you were told. The third has no link, so you ask for one, and what comes back is either a correction or a link that goes somewhere other than where it claimed.

Five minutes, one problem caught, two real studies still in hand. That's the tool working exactly the way it works, and you doing the part that's yours to do.

Where the coach stays in charge

Being suspicious of everything is its own kind of waste. Most of what you get back is fine, and treating every output like a possible lie will cost you more time than the errors ever will.

What matters is knowing where the risk sits. If you're using it to think through something on your own, a small error costs you nothing, because you'll run into it as you work. If you're putting a number or a study in front of your staff, you own it the second you say it out loud.

Match the checking to the stakes. That's the whole skill.

The Coach's Scouting Report

• Signal. A wrong answer looks exactly like a right one. The confidence never changes, so you have to know where the risk lives.
• Noise. Verifying everything. Verifying nothing. Assuming a citation is real because it's formatted like one.
• Next Rep. Next time you get research from AI, open every link before you use any of it. If a claim doesn't come with one, ask for it and see what happens.

See you next Sunday.