Your AI Pilot Doesn’t Have a Job

Somewhere in your organization right now, a team is piloting technology nobody can explain the job of. They can describe the vendor. They can describe the demo that got everyone nodding. Ask them what specific job that technology was hired to do, and watch the pause. That pause is the whole problem. Most AI pilots that fail were never given a job. Nobody wrote down the job description the technology was hired to do, who does that work today, and what would prove it got done better. Writing it down costs nothing. Skipping it costs the pilot. Procurement gave this mistake a name. It doesn’t stay in procurement On a recent episode of the Art of Procurement podcast, Darshan Deshmukh, President of ProcureAbility, gave this pattern its name: “death by a thousand pilots.” His diagnosis was just as plain. Organizations keep leading with the technology instead of the outcome. Philip Ideson, who co-founded Art of Procurement with Kelly Barner and built it into the source procurement leaders actually rely on, put the consequence best in his write-up of the episode: “A compelling platform or use case can be a source of inspiration, but it cannot define the destination.” *(Disclosure: Art of Procurement is an Uphoff Advisory client.)* None of this is new. Procurement made this mistake with ERP, then with e-sourcing, and every other function made it with CRM. Same mistake, different software. The ERP era supplied a half-billion-euro example. In 2018, Lidl walked away from a seven-year SAP project after spending roughly €500 million on it, according to Handelsblatt. The sticking point was not complex. Lidl valued its inventory at purchase price. The software was built around retail price. In my read, nobody settled up front which way the business actually needed that job done, and hundreds of consultants spent years trying to bend one to fit the other. I’ve never run a procurement organization. I have spent three decades running media, marketing, and technology businesses, and every one of them had its own version of this: the CRM migration that died in month four, the martech stack nobody fully turned on, and now the AI copilot every vendor in your inbox is pitching your team. The martech version is measurable. Gartner found marketers used 58% of their martech stack’s capabilities in 2020, 42% in 2022, and 33% in 2023. The spend kept going up. The use kept going down. The failure point is identical everywhere it shows up. The technology got approved. The job never got defined. Why do most AI projects fail? Rarely because of the technology The failure rate for this generation of pilots is well documented. S&P Global Market Intelligence found that 42% of companies abandoned most of their AI initiatives in 2025, up from 17% the year before, and the average company scrapped 46% of its proofs of concept before they reached production. The more useful finding is why. When BCG surveyed 1,000 senior executives in 2024, it attributed roughly 70% of the challenges in getting value from AI to people and process, 20% to technology, and only 10% to the algorithms themselves. The part every demo is built to sell is the smallest slice of the problem. Now look at the budget for your last pilot. If most of it went to the vendor, you funded the 30% and left the 70% to chance. McKinsey’s August 2026 State of AI survey points the same direction from the other end. The companies getting real earnings impact from AI are a small group, about 6% of respondents. Nearly three-quarters of them say they have fundamentally redesigned workflows around it. Among everyone else, the figure is about a quarter. The winners started with the work. Everyone else started with the tool. The business case already exists. It was built backwards Almost every one of those pilots cleared some kind of approval process. Somebody wrote a business case. There were ROI projections, a vendor comparison matrix, maybe a proof of concept demo that went well. This is not usually an absence-of-diligence problem. It’s a diligence-aimed-at-the-wrong-target problem. The business case documented what the technology could do. It rarely documented, in plain words a non-technical person in the business could repeat back accurately, the job the organization actually needed done. Capability and job are different questions. Most business cases answer the first and assume the second. Pilots multiply for another reason nobody puts in the business case. A pilot is the one way to say yes to the vendor and no to accountability in the same meeting. Nobody’s name is on it. Nobody set a date to decide. It costs less than the argument about killing it would. Death by a thousand pilots is death by a thousand decisions nobody made. A job is not a use case I’ve made a version of this argument before, about customers rather than internal technology decisions. When I was brought in to transform ThomasNet.com, my team thought we understood the dozen or two things our customers were trying to accomplish on the platform. The real number, once we did the research, was over 350. We had been building for what we assumed the job was, not what it actually was. I wrote about what that discovery did to the company, and to my own thinking, in the post You Don’t Know What Your Customers Are Actually Trying to Do. The same blind spot shows up on the buy side, and it may be more dangerous there, because nobody outside your own walls is going to do that research for you. A vendor’s use case library describes what other companies did with the tool. It does not describe the job inside your organization, under your specific circumstances, that this technology is being asked to do. If you cannot state that job without naming the technology, you haven’t defined the job. You have named a category. A better demo makes the mistake easier to make An ERP demo in 2004 was a slide deck and a promise. An AI demo in 2026 is the product working on your own data, in your own language, in front of the people who will approve it. The gap between the demo and the job has never looked smaller. It has rarely been larger. A demo shows one job done well, under conditions the vendor chose. Your organization runs hundreds of jobs, under conditions nobody chose. That was the lesson of the 350 at ThomasNet, and no demo is built to show it to you. A good demo makes defining the job feel unnecessary. That is what it was built to do. So I have a rule, and vendors do not love it: no demo until the job is written down. Two companies wrote two different sentences Klarna is the clearest recent example of what happens when the job gets written wrong. In early 2024 the company said its AI assistant was doing the work of 700 customer service agents and handling two-thirds of its customer chats. By May 2025, CEO Sebastian Siemiatkowski told Bloomberg the company was investing in human support again. His explanation was unusually candid: “As cost unfortunately seems to have been a too predominant evaluation factor when organizing this, what you end up having is lower quality.” The job got written as “cut the cost of support.” The job customers needed done was “get my problem resolved.” The pilot did the first job and failed the second. Morgan Stanley did the opposite. Before rolling out its GPT-4 assistant to wealth management, the firm started with a narrow job: help a financial advisor find the right answer, fast, in a library of roughly 100,000 research documents. Every use case was tested against real advisor tasks, with advisors grading the output, before it shipped. According to OpenAI’s case study, more than 98% of advisor teams now use it. Same class of technology as the pilots in the S&P numbers. The difference was a job defined before the purchase order was sent. I wrote the jobs down before I hired a developer When I set out to build the decision tools I use in my advisory work, the easy path was obvious: pick a fast AI development platform, sketch a rough concept, and let a good demo carry the rest. I did the opposite. Before I hired a developer, I wrote out the job to be done for each tool. The valuation tool’s job: when a CEO is weighing a major decision, show what that decision does to the value of the business, and on what timeline. Not “calculate what the company is worth.” That distinction decided what got built. A valuation calculator answers a question once. A decision tool gets checked before every big call, so it needed drivers an operator can actually move, benchmarked against real trading multiples. The talent tools’ job: when a manager has to hire, coach, or restructure a sales or marketing team, help them make that call on evidence, starting with telling a strong resume from a strong hire. Writing that down ruled out the obvious product, an employee-tracking system. The job is the decision, not the monitoring. The technology, the specific platform, the build sequence, all came after those jobs were written down, not before. That ordering is the entire difference between a tool people keep opening and one that stops getting used after the second demo. Agentic AI removed the friction that used to slow this mistake down The old version of this mistake had natural speed bumps built in: long procurement cycles, an IT approval queue, a budget committee that met quarterly. A bad idea had time to die on its own before real money moved. Agentic AI tools mostly skip all of that. A capable pilot can now show up as a subscription one manager puts on a corporate card, no committee required. Zylo’s 2026 SaaS Management Index found expensed software spend up 267% in a year, with ChatGPT now the single most expensed application, and business units, not IT, controlling 81% of software spend. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, and its analysts describe most current projects as experiments “mostly driven by hype.” The old failure mode took a fiscal year to play out. This one takes a sprint. A twenty-year-old mistake now compounds at the speed of a subscription click, and most operators have not updated their discipline to match. The instinct is to add process: a committee, a review, a form. That loses. Governance that costs more than the subscription it governs gets routed around, by the same manager, with the same card. Whatever discipline you add has to be cheaper and faster than the pilot it gates. That is why the test below is three questions and not a forty-page deck. Three gates, cleared before the PO gets signed Here is the test I now run on every technology pitch that reaches my desk, whether I am advising a client or approving something inside my own business. I call it the Application Test: three gates, cleared in order, before a dollar moves. \* **State the job.** In plain words a non-technical person in the business could repeat back accurately, with no vendor name and no product name in it. \* **Name the owner and the baseline.** Who is doing that job today, and how well, in a number you already have: time, cost, error rate, or volume. A committee is not an owner. \* **Set the 90-day signal.** The one measure you will check in ninety days that tells you the job is closer to done, separate from whether the pilot is still running. Logins, seats, and prompts measure the tool, not the job. Decide in advance what you will do with the answer: scale it, fix it, or stop it. Fail any gate, and you do not have a business case. You have a good demo and an open checkbook. Klarna’s pilot would have cleared gate one on cost and failed gate three the first time someone checked whether customers were getting their problems solved. The Application Test decides whether a pilot earns its first dollar. Whether it earns the second is a different discipline, the one I laid out in Your Agentic AI Isn’t Failing. Your Math Is.: never fund the next stage without naming what it has to pay back. Here is an Uphoff Advisory Framework you can download and use in your business. The full worksheet adds the checkboxes, the red flags, and the scoring. Print it, hand it to the team bringing you the next pitch, and ask them to fill it in before the demo, not after. View the full Uphoff Advisory Framework library at uphoffadvisory.com/frameworks. BCG put the algorithms at a tenth of the problem and people and process at 70%. In my experience, most of that 70% starts with a job nobody wrote down. Here is the fastest way to find out whether that is true in your business. Pull up the last three pilots you approved. Write down the job for each one now, after the fact, with no vendor name in it. If you can do it in under a minute, the pilot probably has a future. If you cannot, you already know how it ends. You just haven’t admitted it yet. *The views expressed in Uphoff on Media are entirely my own. They don’t represent the opinions of any company I’ve led, any board I’ve sat on, or any investor who’s had the pleasure of debating strategy with me over the years. If something I write here sounds brilliant, I’ll take full credit. If it turns out to be wrong, I was clearly misquoted by myself.* ***“Uphoff on Media”* is published by Tony Uphoff, Founder and Managing Partner of** *Uphoff Advisory,* *a strategic advisory practice for founders, CEOs, and investors in B2B information, marketing, and technology. The businesses that drive business.* This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit tonyuphoff.substack.com