Why Most AI Projects Stall Before Production: What Six Major Studies Actually Show

Every board deck about AI now carries a failure statistic. "95% of generative AI pilots fail." "80% of AI projects fail." "Over 40% of agentic AI projects will be canceled." The numbers are real, but they come from different studies that measured different things, and most people quoting them have not read past the headline.

This matters if you are about to fund an AI project. The useful question is not "what is the failure rate?" but "what do the projects that fail have in common, and is my project set up the same way?" That is answerable, because the major studies agree on the causes far more than they agree on the percentages.

Below we go through six of the most cited sources, what each one actually measured, and the failure patterns that show up across all of them. We close with the pre-build checklist we use with clients before writing any code.

What the major studies actually measured

Read the numbers carefully

Three caveats change how these figures should be used.

The MIT 95% is about financial impact, not technical failure. Many of those pilots worked as software. They did not move the P&L in a way the company could measure. That is a different and more fixable problem than "the model did not work."

RAND did not measure an 80% failure rate. The report cites that figure from other estimates. Its own contribution is qualitative: structured interviews with practitioners about why things went wrong. That makes it the most useful study on causes and the weakest one on frequency.

Gartner's 40% is a forecast. It is an analyst prediction about agentic projects through 2027, not an observed outcome. It is still worth reading, because Gartner names the specific reasons: escalating costs, unclear business value, and inadequate risk controls. Gartner also warns about "agent washing," the rebranding of existing chatbots and RPA tools as agents, and estimates that only about 130 of the thousands of vendors claiming agentic capability are offering the real thing.

So the honest summary is this: most organizations are not yet getting measurable financial value from AI, a large and growing share are abandoning initiatives, and the reasons are consistent across independent sources.

Five failure patterns that appear across every study

1. The project starts with a technology, not a business problem

RAND's first and most common root cause is that stakeholders misunderstand or miscommunicate the problem AI is supposed to solve. Its third root cause is related: teams chase the newest technique for its own sake. In the report's words, chasing the latest advances in AI "for their own sake is one of the most frequent pathways to failure."

Gartner's "unclear business value" is the same issue seen from the budget side. If nobody wrote down which metric should change, and by how much, nobody can show that it did.

What it looks like: a project brief that says "build an AI agent for customer support" instead of "reduce average handling time on refund tickets from 11 minutes to under 4, without raising the reopen rate."

2. The system cannot learn from the work it does

MIT's explanation for the 95% is not model quality. The authors describe a "learning gap": tools that do not retain feedback, do not adapt to context, and do not fit how people actually work. Users try them, find they have to re-explain the same things every time, and go back to the old process.

This is an engineering problem with engineering answers: capturing corrections, storing approved outputs as examples, keeping per-customer context, and measuring quality over time instead of at launch only.

3. AI is added to the old workflow instead of changing it

McKinsey's high performers, the roughly 6% of respondents attributing 5% or more of EBIT to AI, are nearly three times as likely as others to have fundamentally redesigned their workflows. BCG reaches the same conclusion from a different angle. Its "10-20-70" rule says leaders put about 10% of effort into algorithms, 20% into technology and data, and 70% into people and processes, and that about 70% of implementation problems come from people and process issues.

What it looks like: an AI tool that drafts a document, which then goes through the same five approval steps it went through before. The drafting got faster. The process did not.

4. The data and infrastructure are not ready

RAND lists two causes here: organizations lack the data to train or ground an effective system, and they lack the infrastructure to manage that data and deploy the result. With large language models the first problem looks different (you rarely train a model from scratch), but it has not gone away. A retrieval system grounded on outdated, duplicated or contradictory documents will give outdated, duplicated or contradictory answers.

5. Cost and risk controls are added after launch

Gartner names escalating costs and inadequate risk controls as two of its three reasons for cancellation. Both tend to appear late. Token costs that looked trivial in a pilot with 20 users become a line item at 20,000. Permissions, audit trails, and human approval for high-impact actions get designed after something has already gone wrong.

A sixth pattern: too many pilots at once

BCG found that leading companies pursue about half as many AI opportunities as their less advanced peers, and they concentrate on the few with the largest expected return. Spreading a budget across fifteen proofs of concept almost guarantees that none of them gets the integration and workflow work needed to reach production. S&P's data points in the same direction: the average organization scrapped 46% of its AI proofs of concept before they reached production.

Why technically strong projects still fail

The studies above share one uncomfortable implication. In most failed projects, the model was not the bottleneck. The model usually worked well enough. What failed was the definition of success, the integration with real work, the feedback loop, or the economics.

That is also why "we will just use a better model" rarely rescues a stalled project. A more capable model does not fix a missing success metric, a workflow nobody redesigned, or a knowledge base nobody maintains.

A pre-build checklist

Before we write code on an AI engagement, we ask the client to answer the questions below in writing. If more than two answers are "we don't know yet," the right next step is a short discovery phase, not a build.

The last question deserves its own article, and we wrote one: How to evaluate an LLM feature before it reaches customers.

The bottom line

The failure statistics are not an argument against AI projects. They are an argument against starting AI projects the way most companies start them: with a technology choice, a broad mandate, an unchanged workflow and no plan for measurement.

The organizations getting returns are doing fewer things, choosing them by expected value, changing the work around them, and treating quality and cost as engineering requirements from the first week. None of that requires a bigger budget. It requires deciding what success means before the build starts.

If you want help answering the checklist above for a specific project, our two-week Discovery Sprint produces exactly that: a scoped problem, a baseline metric, an architecture and a fixed-price plan. You can also get a rough cost range from our project cost calculator.

Sources

• MIT NANDA, "The GenAI Divide: State of AI in Business 2025," as reported by Virtualization Review
• RAND Corporation, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed, Ryseff, De Bruhl and Newberry, August 2024
• S&P Global Market Intelligence 2025 survey, as reported by CIO Dive
• Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, June 25, 2025
• BCG, AI Adoption in 2024: 74% of Companies Struggle to Achieve and Scale Value, October 2024
• McKinsey & Company, The State of AI in 2025: Agents, Innovation, and Transformation, November 2025