The cause sits upstream: unread intent, missing oversight, thin context, loose language, unset expectations, no evals, fuzzy outcomes, and the wrong tool for the job. Here’s the AI checklist.
Anthropic recently told its growth team to hire more product managers, not fewer. You’ll be surprised at the reason why.
Claude Code had turned its engineers into a team shipping at triple its headcount, and the bottleneck moved from writing code to deciding what to build, which is why everything is a research preview—so they can get user feedback.
A Jevons Paradox moment to say the least.
The technology no longer the constraint; it now takes more time to plan and get to a shared understanding, according to Microsoft’s Travis Lowdermilk, than to build. We’re moving so fast, misalignment is the outcome, which could lead to project failure.
What’s new is the acceleration of code generation. AI doesn’t create the need for shared understanding. It puts even more pressure on it.
— Travis Lowdermilk
Because of this they want the humans to be aligned and in control, which is a perfect time for user experience: put in enough thought to do it right.
Right now when a project stalls, the instinct is to blame the model and wait for the next one. It rarely helps; a 2025 MIT study found that roughly 95 percent of generative AI pilots deliver no measurable impact.
This is not new, just has a different target since it didn’t take AI to have failed software projects that cost a lot of money—it’s always fun to link to CIO for that—and in times of innovation this is to be expected because we underestimate the ambiguity we because we don’t have a shared perspective. All white space that happens in meetings and Slack channels, and not in code.
We’re still in early times so it’s also a learning and expectations gap: a capable tool that has known constraints inside an organization that hasn’t changed, AI or not.
Most AI failures are organizational, not technical—Ironically a failure of human in the loop making unrealistic decisions.
I have spent 11 years around AI, the last three and a half in generative AI. We built these systems and I have found that the bottleneck sits upstream, in the scaffolding around it: the user’s intent, who is accountable, what the model can see, how the work is specified, what the system promises, whether you can measure it, the outcome you are chasing, and the tool you picked. Not a single line of code in there as the reason.
And frankly, a lot of the above are just solid software development practices that are even more important in the era of AI.
Eight human problems, each wearing a technical hat. None waits on a better model or line of code; they wait for better human decisions.
The machine is ready to run. No one has written down what for.Intent: Start From What People Are Trying to Do
Most AI efforts start from the capability and work backward to a use. The model can summarize, so we add a summarize button. It can chat, so we bolt on a chatbot. Nobody asked what the person was trying to accomplish; the technology proposed itself, and the team said yes.
That is how you ship a feature no one opens, and it’s the most basic of user experience first principles i.e. what are you solving for.
The discipline is to start from intent: the job the person is trying to do, in their words, before any mention of AI. I have written about what Will Manidis calls a tool-shaped object, something that produces the feeling of work without the work.
Capability-first AI is a factory for them: it feels productive and ships nothing anyone needed, but it looked great in a demo.
Nobody asked what the person was trying to accomplish; the technology proposed itself, and the team said yes.
Intent is what every later decision hangs on. You cannot match the tool to the job, set honest expectations, or define an outcome until you can state, plainly, what the person is here to do.
Get the intent wrong and the most capable model in the world will execute the wrong task perfectly. Start there, in plain language, before anything else.
Action items
- Write the intent as the user’s sentence. State what the person is trying to accomplish in their words, with no mention of AI, before you design anything. How might we’s are a great way to start so there’s alignment.
- Kill capability-first ideas. If the pitch starts with what the model can do rather than what someone needs, send it back until it names the job. That’s technology looking for a problem.
Fit: Use an Agent Only When the Work Needs One
The fastest way to fail with AI is to reach for the most powerful version of it. I call it flying down with a 747 to the corner store to grab a gallon of milk.
An autonomous agent that plans, calls tools, and acts on its own is impressive and expensive and hard to oversee. Most of the work people point it at is retrieval, summarization, or a fixed sequence of steps, jobs a simpler tool does more reliably in a deterministic approach and at a fraction of the cost and risk.
Gartner’s analysts have been direct: many of the use cases being sold as agentic today do not need an agent at all, and the firm estimates only about 130 of the thousands of vendors claiming agentic capability are the real thing.
The industry has a name for the rest: Agent washing.
Buying it means paying agent prices for assistant work. Slower, less accurate and more expensive than before.
The fastest way to fail with AI is to reach for the most powerful version of it without understanding the need.
The fix starts with a clear-eyed look at the task. Fit the tool to the job: use an agent when a decision needs to be made, automation when the steps are fixed, and an assistant when someone just needs an answer.
And when the work really is agentic, the next question is where it should live.
- Chat fits exploration, one-off questions, and tasks where the person wants to steer turn by turn.
- Ambient fits the rest: work that runs in the background, triggered by an event, surfacing only when it needs a decision or hits a limit you set.
Forcing everything through a chat box is the common error, so an agent that should run quietly ends up waiting for someone to open a window and type. Match the modality to the moment, and use each where it fits.
Reach for a background agent only when the task clears this bar. The first four decide whether the work is agentic at all; the last two decide whether it is safe and worth it.
- A real decision under uncertainty. The right next step is not known in advance. If you can draw the flowchart, it is automation, not an agent.
- Steps that depend on each other. The work runs several steps where each shapes the next, so the plan has to adapt as it goes rather than follow a fixed script.
- It has to act, not just answer. The task calls tools or takes actions in a system, not only returns text.
- Inputs too varied to hardcode. The situations differ enough that a fixed workflow would break or need constant patching.
- A bounded, overseen blast radius. Consequential actions sit behind permissions and a human checkpoint, so a wrong move is recoverable.
- Autonomy that earns its cost. The time or scale the agent saves outweighs the expense, latency, and risk of letting it run on its own.
Action items
- Match the tool to the task. Start with the MVP. Decision needed, agent; fixed steps, automation; retrieval, assistant. Most work is the last two.
- Match the mode to the moment. Use chat for exploration and turn-by-turn steering; run ongoing or background work ambient, on triggers, instead of behind a chat box the user has to remember to open.
- Demand proof of autonomy. If a vendor sells an agent, ask what it decides without a human; if the answer is nothing, it is not one.
Expectations: Show Users What to Give It and How Far to Trust It
A new hire gets an onboarding: here is what you will handle, here is what to bring to it, here is how sure you should be before you act. Most AI systems give the user none of that, on either side. They rarely say what to feed them to get a good result, and they return a confident paragraph whether the answer is solid or a guess.
The user is left to guess at the input and calibrate trust in the output with no signal for either.
Expectations run both ways. On the way in, a good system shows what a strong request looks like, what context to include, and what it will quietly ignore. Most do not, so a user types a vague line, gets a vague answer, and concludes the model is weak when the real gap was the prompt. Telling people what to give you is the cheapest quality lever there is.
Telling people what to give you is the cheapest quality lever there is.
On the way out, the same honesty applies.
The team behind Microsoft’s Guidelines for Human-AI Interaction, Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Eric Horvitz, and colleagues, put two rules first:
- Make clear what the system can do
- Make clear how well it does it
Most products ship the capability and hide the limits, then act surprised when users over-trust a wrong answer or abandon a useful one.
The over-trust is well documented: in one controlled study, trained pathology experts overturned their own correct judgment to follow the AI’s wrong call in about 7 percent of decisions, a pattern researchers call automation bias, and a system that never signals its own uncertainty makes it worse.
We’re still learning about how accurate the models are, which is good considering my reported passing as of March 2026 in GPT-5. It has since been fixed. However, that’s an expectation we have to set — we have to check for accuracy. Human in the loop.Setting expectations is mundane work on both ends, which is why it gets cut. Tell people what to bring. Show explainability. Name what the system cannot do before they find out the hard way.
None of it is glamorous, and all of it is the difference between a tool people trust and one they quietly stop using, mainly because they don’t understand the constraints.
Action items
- Tell users what to give it. Show what a good request looks like, what context to include, and what the system will ignore, so a vague input does not become a vague output you both blame on the model.
- Tell users what to trust. State what the system does well, does poorly, and how often it is wrong, and make an uncertain answer look different from a sure one.
Oversight: Decide What the AI Is Allowed to Touch
Most AI deployments have no owner. There is a sponsor who approved the budget and a vendor who sold the tool, but no single person accountable for what the system decides and what happens when it decides wrong.
That vacuum is fine in a demo; it is not fine when the agent has write access to customer records.
Oversight is not paperwork, it is the harness: the set of permissions that decides what the AI can and cannot do, so that doing the wrong thing is never available to it.
The most reliable way to stop an agent from taking a harmful action is to make sure it never had permission to take it. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027, and one of the three named causes is inadequate risk controls.
A risk control is an oversight decision before it is a technical one. What is the agent allowed to touch? Who reviews its actions? Who can shut it off?
The most reliable way to stop an agent from taking a harmful action is to make sure it never had permission to take it.
This is not hypothetical. In 2025, Replit’s AI coding agent deleted a live production database during a code freeze, wiping records for more than 1,200 executives and 1,190 companies, despite repeated instructions to change nothing. I had a conversation with an UX expert this, and we aligned that it should have never happened because you have to put the same guardrails in for AI that you have to for people.
The model was not the failure.
The failure was that an agent in development could reach production and run a destructive command at all, which is a security problem. Replit’s own fix was to enforce separation between development and production like any normal system, an oversight decision made one disaster too late.
When those answers are missing, the project does not fail loudly. It stalls in a holding pattern, never trusted enough to leave the pilot. Security teams have known the fix for decdes under a plain name: least privilege.
Give the system the narrowest access that still lets it do its job, and the exposure of any mistake shrinks to something you can live with. Before you tune a prompt, decide what the AI is permitted to do just like you would do for a human, and name someone who owns that decision.
Action items
- Scope permissions to least privilege. Grant the narrowest access the task requires; everything the agent cannot reach is a harm it cannot cause.
- Name an accountable owner and a stop condition. One person answerable for the system in production, and a documented way to halt it, settled before the first deployment rather than after the first incident.
Context: Give the Model What It Cannot See
A model knows what is in the prompt and what is in its training. It does not know your pricing exceptions, your renewal calendar, the customer who threatened to leave last week, or the unwritten rule that finance signs off on anything over ten thousand dollars.
The MIT researchers found that pilots stall on brittle workflows and a lack of contextual learning, where the tools never absorb how the work is done because they don’t have context of the white space.
That is not a model defect. It is a context-supply problem, and the supply chain runs through your documentation, your data, and your willingness to expose both. It fails, because the machine gave answers no one would trust — it was playing blackjack when it couldn’t see either card the dealer was holding.
When the output is generic, the model is not failing. It is answering the only question it could see. This is the part most teams skip because it is unglamorous.
This is where I see most projects failing most — not enough context for the AI to hang its hat upon. I’m not alone in saying this.
Andrew Ng has spent years arguing that the fastest gains in applied AI come from improving the data, not the model. The same logic holds for context. A mediocre model with rich, current context will outperform a frontier model working blind, every day of the week.
Knowing full well every agent is a context experiment for what it is good enough, this is going to take time to learn. Until then, feed it the organization as much as possible, or accept that it is guessing.
Action items
- Inventory what the model can see. Write down the context it has and the context it needs; the gap is your roadmap.
- Put context what the model can’t see. A document no system can retrieve is context the model does not have. Then you know where the risk is.
Language: Write Instructions Tight Enough to Execute
Ask three people to summarize the customer feedback and you will get three different documents. Ask a model the same thing and you get a fourth. The problem is not comprehension. The problem is that the instruction never specified length, audience, format, what to include, or what to leave out. Ambiguous in, ambiguous out.
We spent decades teaching teams to write clear requirements for software that executes literally. AI is different in kind.
It is probabilistic, not deterministic: it predicts the most likely next words rather than running your instructions to the letter, and sometimes it will not get it right, literally by design. The model fills any gap you leave with a plausible guess, which is whywhy controlled language and controlled vocabularies, precise and structured and unambiguous, is now a core skill rather than a nicety. Most people don’t have that skill, acting like Michael Stipe at an R.E.M. concert.
The prompt is a specification, and most prompts are underspecified.
This is where a consistent structure across most of your documents earns its keep: A request with named fields, explicit constraints, and worked examples outperforms a paragraph of hopeful prose every time.
The discipline is old, and it is why you keep UX professionals around.
Technical writers, information architects, and content designers have spent years turning fuzzy intent into precise, structured language, which is exactly what a good prompt demands. Yuval Keshtcher and the content design field treat this as craft: designing the rules that govern language, not writing one string at a time.
AI just made that craft foundational for everyone else.
A controlled prompt is built, not typed. I use a format I have written about, CARE: Context, Ask, Rules, Examples. Context is the background the model needs. Ask is the request in one clear sentence.
Rules are the constraints it must respect, including format and tone. Examples show what good looks like. Leave any of the four blank and the model fills it with a guess. Name all four and the output stops drifting, because there is nothing left to infer.
Action items
- Write the prompt in CARE format. Context, Ask, Rules, Examples; a blank in any one is where the output starts to drift.
- Specify before you prompt. State the audience, length, and constraints up front; do not make the model infer them.
Evals: Measure Whether It Works Before You Ship It
Most teams cannot answer a simple question about their AI feature: is it better or worse than last week?
They shipped on a handful of prompts that looked good in a demo, and they have measured nothing since. So when they change the model, tweak the prompt, or swap a vendor, they are guessing.
The system might have improved.
It might have quietly broken.
Nobody can say.
Evals are the fix: a fixed set of real inputs with known-good outputs, scored the same way every time, so a change produces a number instead of a feeling. State what success is before you start.
Conventional software has had this for decades; we call it a test suite. I know, because I was leading test driven development as a Product Manager in 2002, and it acc