Don't let your judgment detrain

The better it gets, the easier it is to stop checking

Everything you hear about AI right now is about how good the models are getting, and most of it is true. So the natural move, especially if you're newer to the field, is to go straight to the tool, get an answer, and trust it. When you're busy and the answer sounds confident, there's not much reason to question it.

I think that habit is the biggest risk in this whole space. The tool being wrong now and then is manageable. What worries me more is what happens to you when you stop doing the thinking yourself.

What the research actually says

A study out of Carnegie Mellon and Microsoft surveyed 319 people who use AI in their work every week and collected almost a thousand real examples of how they used it. The pattern they found was simple. The more confidence someone had in the AI, the less they thought critically about what it gave them. The more confidence someone had in their own ability, the more they checked, refined, and pushed back.

They found something else that matters even more for coaches. People tended to skip the critical thinking when they didn't have the skills to judge the output in the first place, which means you're at your best catching the tool's mistakes in the areas where you could do the work yourself.

It's worth being honest about what this is. It's a survey of how people described their own habits, and it wasn't a study of coaches. A review of AI research in sports medicine published this summer raises the same concern for our field, the idea that leaning on these tools too much could wear down the skills you need to catch their errors, and the authors are clear that it hasn't been tested in practitioners yet. So I'd treat it as a risk worth managing rather than a proven effect.

Think about it like detraining

Every coach already understands this, just in a different context. An athlete adapts to the demands you put on them, and when you take the stimulus away, the adaptation fades. Nobody expects an athlete to hold their speed through twelve weeks of no sprinting.

I think judgment works the same way. If AI does all of your data interpretation for a season, you shouldn't expect to read a jump report as sharply at the end of it as you did at the start, because the skill stopped getting reps.

So I keep three things for myself. I make the decisions, I write the programming, and I interpret the data. AI helps with almost everything around those three things, the formatting, the summaries, the research gathering, the documents, the organizing. But those three are the job, and they're also exactly the skills I need in order to know when the tool got something wrong.

Inside the Lab: think first, then bring it in

The simplest way to keep your reps is to change the order you do things in. I call it Think, AI, Decide.

Think first. Before you open anything, form your own take. What does the data say to you? What would you program? What's your read on the situation? Two minutes of rough notes is plenty.

Then bring in AI to pressure test it. Instead of asking for the answer, hand it your answer and ask it to push back. Something like this:

"Here's my read on this and what I'm planning to do. Don't give me your own answer first. Tell me where my reasoning is weakest, what I might be missing, and what information would change the call. Then tell me where you'd disagree with me and why."

Then you decide. Keep what holds up, drop what doesn't, and make the call yourself.

The difference between those two approaches is bigger than it looks. When you ask for the answer first, your job becomes approving it, and approving something is a much lighter rep than thinking it through. When you bring your own answer first, the tool becomes a training partner you have to argue with.

What it looks like in the real world

Say you've got a week of jump data and wellness scores in front of you. The fast version is pasting it in and asking what it means. The better version is looking at it yourself first, writing down who you're worried about and why, and then asking the tool what it sees.

Where you agree, you've confirmed your read. Where you disagree is where it gets useful, because now you have to work out which of you is right, and either way you come out of it sharper. Working through that disagreement is the rep, and it's the exact part you skip when you ask for the answer first.

Where the coach stays in charge

To be clear, I still think most coaches should be handing off more busywork than they are, and that's what this whole newsletter is about. The line is between busywork and judgment, and every coach's line is going to sit in a slightly different place.

If you manage GAs, interns, or young assistants, this matters even more. They haven't built the skills yet, which makes them the most likely to trust an answer they can't check. Have them do the read first before they ask the tool.

The Coach's Scouting Report

• Signal. You're at your best catching AI's mistakes in the areas where you could do the work yourself, so keep the reps in those areas.
• Noise. Asking for the answer before you've formed your own. Trusting an output because the models are supposed to be good now.
• Next Rep. Write down the three things in your job you won't hand off to AI. Then next time one of them comes up, form your own take first and ask the tool to argue with it.

From the Lab

If you or someone on your staff is still working out where that line should sit, the AI Starter Guide for Strength Coaches is a $20 ebook that covers where AI fits in your week and what to hand off first.

See you next Sunday.