Every leader is asking the same question: which jobs will AI take? It's the wrong question. The better one is narrower, and it predicts far more: which of your tasks produce an output you can measure?
That single distinction explains most of what I see when I watch AI succeed in some places and stall in others. It's not about how hard the task is, how skilled the worker is, or how much data exists. It's about whether the result can be checked against a clear standard. Where the output is easy to measure, AI will get there. Where it isn't, AI struggles—no matter how impressive the demo looks.
Here's why, and what it means for where you point AI and where you protect people.
Why software fell first
Start with the one domain where AI has moved fastest: writing code.
It's tempting to think software went first because programmers are technical, or because there's a lot of code on the internet to train on. Those help. But the real reason is quieter: software has a built-in answer key. A unit test either passes or it doesn't. The output is unambiguous, and it's cheap to check.
That changes what AI can do. When the output is measurable, an AI system can try, check itself, fail, and try again—thousands of times, without getting tired. It iterates against the test until it passes. The measurable output isn't just how we grade the work at the end; it's the thing that lets the AI improve in the first place.
Take away the answer key and that loop collapses. The model can still produce something. It just can't tell whether the something is any good, and neither, quickly, can you.
The lesson from machine learning
I spend a lot of time in data science, and this is an old lesson there, wearing new clothes.
You cannot build a machine learning model to predict something you can't clearly define. In the trade we'd say you have a "Y data problem" — your outcome variable is ambiguous, inconsistent, or contested, so there's nothing stable for the model to learn toward. Give a model a fuzzy target and it will confidently learn the wrong thing.
Generative AI hasn't repealed that constraint; it's inherited it. An agent that "keeps working until it gets it right" needs a definition of right it can measure. Give it one and it's remarkable. Withhold one and it will still hand you an answer, delivered with the same confidence—which is exactly the problem.
So the question isn't whether a task is complex. Complex tasks with clear answers automate well — think facial recognition. The question is whether the target is well-defined enough to check.
A test you can run on any task
Put your work on a spectrum. At one end, tasks with a crisp, cheap-to-verify output. At the other, tasks where "good" is a matter of judgment, context, and nuance—where two experts might reasonably disagree.
Near the measurable end: reconciling accounts, extracting fields from documents, converting a dataset from one format to another, generating code that has to pass tests, routing a standard support ticket. You know immediately whether the result is right. These are exposed to automation, and pretending otherwise won't slow it down.
Near the ambiguous end: setting strategy, positioning a brand, coaching an underperformer, deciding which of three good candidates to hire, judging whether an argument is persuasive to this board. There's no answer key. The output is contested, the feedback is slow, and "correct" depends on things that are not written down. AI can draft into these tasks, but it can't close the loop on them—because no one can hand it a test it can run.
Most real jobs are a mix. The useful exercise isn't labeling whole roles "safe" or "doomed"; it's going task by task and asking: could I write down, cheaply and unambiguously, what a good result looks like? If yes, expect AI to move in. If no, that task is where your people remain valuable.
The measure is not the goal
There's a catch, and leaders fall into it constantly.
When a task's real output is hard to measure, the temptation is to substitute a proxy that is easy to measure—and then automate against the proxy. Measure a support team on tickets closed, and AI will close tickets, whether or not the customer's problem is solved. Measure writing by word count or reading level, and you'll get text that hits the number and misses the point. Measure teaching by information delivered and you can automate the delivery while quietly abandoning the part that was hard to measure and mattered most.
This is Goodhart's law: once a measure becomes a target, it stops being a good measure. AI makes it faster and cheaper to optimize a bad proxy to its limit. So before you automate against an output, make sure the output you're measuring is the one you actually want—not just the one that was convenient to count.
But there's a subtler failure, and it shows up even when you've measured exactly the right thing. An optimizer doesn't pursue your intent; it pursues the measure—and the gap between the two is never zero. Give an AI a genuine, well-chosen test and it may learn to pass the test rather than do the work: hardcoding the expected answer, weakening the check, or finding some path that turns the metric green without solving the problem. This is reward hacking, and it's routine behavior, not a rare glitch. Recall the unit test from earlier. The same measurable output that lets AI iterate toward correct is also a target it can game. A passing test tells you the measure was met. It does not, by itself, tell you it was met for the reason you cared about.
Both traps share a root: the measure is a stand-in for the goal, and AI optimizes the stand-in. Choosing a good measure protects you from the first. Only verification—checking that the result is right, not just that the number is green—protects you from the second.
What this means for how you lead
Three practical implications.
First, a map of automatable tasks is not a plan. The measurable-output test tells you where AI can take over. The temptation is to treat that as a checklist—unbundle the job, pick off the measurable tasks, declare victory. But a job isn't a pile of independent tasks; it's a process, and the pieces are intertwined. Automate one measurable task inside a workflow built for the old constraint and you get a faster task and an unchanged result. I've written about this as the new productivity paradox: the analyst who turns a three-week report around in two days doesn't speed up a company whose decisions still move at the speed of the monthly meeting. The gain leaks out in the seams between tasks. Capturing it means redesigning the process around what AI made cheap—not paving the old path faster.
Second, where you want automation, invest in defining the output. Much of the work of getting value from AI isn't prompting; it's the unglamorous labor of writing down what "good" means clearly enough to check. Teams that do this get compounding returns. Teams that skip it get confident, plausible, unverifiable output — workslop that undermines trust in the tool.
Third, where the output is genuinely ambiguous, lean on human judgment. These tasks resist automation not because AI is weak but because the target is contested, and that won't change soon. This is where human expertise stays scarce and valuable. Defending that work—and refusing to paper over it with a convenient proxy—is a strategic choice, not a nostalgic one.
AI automates against a measurable output. That's the whole test. It tells you where to push, where to wait, and where to keep your best people pointed. The organizations that thrive won't be the ones that automated the most. They'll be the ones that knew the difference between what could be measured and what couldn't—and didn't confuse the two.