Hey there,
I had a coaching call recently with a client who walked me through something he'd been working on, and it hadn't gone the way he'd hoped.
He'd done the painstaking work of mapping out a complicated process. It's a compliance process, in healthcare, something his organization gets audited on regularly and sometimes with no warning, so getting it right isn't optional. He'd put in the research, the interviews, the digging it takes to document how something actually runs rather than how it's supposed to run. All of that turned into a highly accurate and detailed current-state flowchart.
He wanted to see where and how this complicated process could be simplified so he uploaded the flowchart to Claude and asked where the process could be improved and tightened. What came back was reasonable, but completely generic. There wasn't anything particularly wrong with it, but there wasn't anything in the response that he didn't already know. And he'd taken that as a limitation of the tool.
So I asked him: before you uploaded the flowchart to Claude, did you already have a sense of which parts of the process were problematic?
Of course he did. Because of his years of experience combined with all the time he'd just spent digging into this workflow, he knew which areas were bloated and which handoffs didn't have a good reason for existing anymore. That knowledge was the real output of his diagnostic work. The flowchart was just the part he could put on a screen and show to other people.
And when it came time to bring AI into it, he handed over the flowchart and inadvertently kept the knowledge to himself.
So the model got a map with no destination on it, attached to a question with no aim in it. Where can this be improved?There's no good answer to that, only a generic one, because the question is generic. The tool wasn't underperforming. It answered precisely the question it was asked.
What I suggested instead was to stop feeding it the whole process. Take the specific areas he already suspected, and go at them one at a time, bringing everything he knew along with them. Here's this piece of the process. Here's why I think it's more complicated than it needs to be. Here's what I know about it. Based on this, how do you think this could be improved?
That's a different conversation, and it produces a different quality of answer, because now the AI has something to push against.
“He handed over the flowchart and inadvertently kept the knowledge to himself.”
What I keep coming back to and, frankly, why I think this anecdote is worth writing about, is why this mistake is so easy to make and so hard to catch in yourself.
Most of the writing about AI and thinking is some version of a discipline argument. Do your own thinking first. Don't outsource your brain. Resist the shortcut. All of that assumes the person is cutting a corner and probably knows it, and needs to be talked out of it.
That isn't what happened here. My client didn't skip a step. He did more diagnostic work than most people in his position would have, and he did it well. He produced a serious piece of work through serious effort, and then he handed that serious piece of work to the AI, because handing over the most complete thing you have is what responsible people do. It didn't feel like a shortcut. It felt like the right next step.
I've started thinking of this as the artifact handoff. You do the hard work, the hard work produces something visible, and somewhere along the way you start treating the visible thing as if it were the valuable thing. Then you pass the visible thing along and leave the valuable thing sitting in your own head, where it does nobody any good, including you.
This isn't new, and it isn't really about AI. We've done it to each other for years. You hand a colleague the deck instead of the argument that produced it. You hand a new hire the process documentation instead of the reasons the process ended up looking the way it does. AI just made it faster, and it hands you back something articulate enough that it takes a minute to notice how little is in there.
None of this squares with something you probably already know, which is that handing a big document to AI is often exactly the right move. Give it a contract and it will find the clause that's going to hurt you. Give it a long RFP and it will come back with a summary tighter than one you'd have written yourself. The handoff of a large document works. So why didn't it work here?
The easy explanation is that a flowchart is a picture, and models read pictures less reliably than they read text. There's a sliver of truth in that, but it isn't the reason. Write the whole flowchart out in clean prose, every box and arrow described, and the same empty answer comes back. The format wasn't the problem.
Handing off the contract works because it contains its own answer. Everything needed to spot the risky clause is written on the page. The document is complete for the question being asked. My client's flowchart was complete too, but only as a picture of what the process is. It said nothing about what was wrong with it, because that part was never written down anywhere. Which steps were bloated, which handoff had outlived its reason, what "better" would even mean here, all of that lived in his head. He built that understanding in the act of making the flowchart, and almost none of it made it into the flowchart itself.
So the criterion that mattered was never the size of the artifact or even the format of it. It's whether the artifact actually holds what's needed to answer your question, or whether the answer lives in something you know but never wrote down. Summarize this, pull the key terms, tell me what I'm missing in this contract, a document can answer those on its own. Where should this be improved cannot, because improvement is a judgment, and the judgment was in his head the entire time.
That's the difference between using AI as a partner in diagnosis and using it as a replacement for it. And you can't tell the two apart by looking at the output. Both come back organized and confident. The only thing that separates them is what you put in, and you're the only one who knows what that was.
My client is going back through those areas this week, one at a time, feeding in the specific parts and everything he knows about each of them. He already did the part that matters. Now the tool can actually help him finish it.
Break a Pencil,
Michael (breakapencil.com)
P.S. If someone on your team keeps handing the whole thing to the AI, forward this along.
P.P.S. Where have you handed over the artifact when you should have handed over what you knew? Hit reply, I read every one.