AI Pilot to Production: How to Rescue, Restart, Buy or Stop a Stalled Pilot

Most companies that started an AI pilot in 2024 or 2025 have one in the same state today. The demo impressed the leadership team, a small group kept it alive, and nobody has signed the decision to run it for real. In a Zapier survey of more than 800 senior leaders, 84% had at least one AI pilot that never reached production, and 38% said their longest-running pilot had been in evaluation for over a year.⁹ The explanation people reach for is a statistic, usually that 95% of AI pilots fail. That figure is weaker than its fame, and the stronger numbers point somewhere more useful. Reaching production is close to a coin toss. Paying back once there is the hard part, and both outcomes turn on choices made before and during the pilot.

• 48%of AI projects reached production in Gartner's survey of 644 organizations, about eight months after the prototype¹
• 6%of organizations attribute 5% or more of EBIT to AI, a share that has not moved in two years³
• 5xGartner's predicted rise in inference cost per agentic workflow through 2028, while token prices keep falling⁶

How many AI pilots actually reach production

"How many AI pilots fail" hides two questions, and the evidence answers them very differently. The first is whether a pilot becomes a running system. Gartner surveyed 644 organizations in the US, Germany and the UK at the end of 2023 and found that on average 48% of AI projects made it into production, about eight months after the prototype.¹ S&P Global's 451 Research surveyed 1,006 professionals in North America and Europe a year later and found the average organization ended 46% of its proofs of concept before production, which leaves a little over half going through.¹⁰ IDC's 2026 CIO Playbook for Lenovo, based on 3,120 decision-makers surveyed in late 2025, reports that 46% of AI proofs of concept progressed into production.² Deloitte sets a stricter bar and gets a lower number: 25% of 3,235 leaders say their organization has moved 40% or more of its AI pilots into production.¹¹

The second question is whether the system pays once it runs, and here the answer is about one in twenty. McKinsey's State of AI 2026 found 44% of respondents reporting AI scaling across the enterprise, while only 37% attribute any EBIT impact to AI and most of those put it under 5%. Its high performers, who attribute 5% or more of EBIT to AI, have stayed at about 6% for two years.³¹² BCG's 2025 study of 1,250 executives puts 5% of firms in its "future-built" group and 60% getting hardly any material value.¹³ Gartner's 2026 survey of 1,303 organizations found 22% have scaled AI across multiple business units, and 11% could not say what their function had spent on AI the year before.¹⁴

Read together, the numbers move the bottleneck. Getting from pilot to production succeeds about half the time. Getting from production to a result the finance team recognizes succeeds for a small minority, and that minority has not grown while adoption has. Every figure here is self-reported or an analyst's prediction, and no one independently audits enterprise AI outcomes, so treat them as the best available signal and plan with them on that basis.

What the famous failure numbers measured

Three statistics dominate this topic, and each one fails the simplest test a CTO can apply: what did the source actually count? The table sets each widely quoted figure beside what its publisher measured.

The MIT report deserves a closer look because it shaped a year of coverage. Its own exhibit carries the authors' caveats: the numbers are directional, they come from individual interviews, and six months may be too short to judge complex systems.⁴ Ray Poynter pointed out that if 20% piloted and 5% cleared the bar, roughly one piloting organization in four succeeded.¹⁸ Wharton's Kevin Werbach said he could not find where the 95% came from.¹⁹ The report still has useful findings, including the "learning gap" of tools that never retain feedback, and the observation that workers at more than 90% of firms use personal AI tools while only 40% of firms bought an official subscription.⁴

A defensible sentence replaces all three headlines. In the largest surveys roughly half of AI proofs of concept reach production, and about 6% of organizations attribute 5% or more of EBIT to AI. Leading a board paper with "95% fail" tells anyone who has read the source that the paper's author has not.

Why pilots stall between the demo and production

The research on causes points at organizations and data far more than at models. RAND's five root causes are a misunderstood problem, missing data, technology chosen before the user's problem, inadequate infrastructure to manage data and deploy models, and problems too hard for current AI; 84% of its interviewees named a leadership-driven cause as the main one.⁵ Gartner's four reasons for abandonment are poor data quality, inadequate risk controls, escalating costs and unclear business value.¹⁶ In a separate Gartner survey, 63% of organizations either lacked the right data management practices for AI or were unsure whether they had them.²⁰ Zapier found infrastructure, data quality and integration problems blocking 41% of stalled pilots and legal or compliance concerns blocking 29%, and organizations that deploy most of their pilots were twice as likely to have an executive sponsor.⁹

Deloitte describes the structural trap well. A small team can run a pilot in a few months on cleansed data in an isolated environment. Production needs infrastructure, integration with existing systems, security reviews, compliance checks, monitoring and maintenance, and failures that were learning opportunities in the pilot become business risks once real customers depend on the system.²¹ A pilot built on an extract, with a broad service account and no cost ceiling, proves the model can answer. It says very little about whether the system can run.

Adoption kills pilots more quietly. The shadow-AI finding shows the demand is real, and a sanctioned tool that cannot remember context, learn from corrections or reach the right data loses to the consumer chatbot people already use.⁴ Licensed assistants show the same pattern: in a 2025 Gartner survey of IT leaders, only 5% of organizations that had finished a Microsoft 365 Copilot pilot were moving to a larger deployment.²²

What production asks of a system that a demo never did

A pilot proves a model can give good answers on a curated set. Production has to prove the whole chain of retrieval, prompts, model, tools and gateway holds its quality on live inputs, survives attack, stays inside quota and budget, and keeps working when the provider changes the model underneath it. Microsoft's architecture guidance warns that a team can be mature at MLOps and a beginner at operating generative AI, because the governed system now includes the orchestrator, the calls behind it and the prompts it builds.²³ Google's guidance asks for evaluation of whole chains, checks for skew between evaluation inputs and production inputs, and strict versioning of every component that can change.²⁴ Our evaluation guide covers how to build that test suite.

The model row surprises most teams. On Microsoft Foundry, a generally available model version retires 18 months after launch, every request to it fails from that day, and the official replacement is named only about 90 to 120 days before.²⁵ Providers also regress without warning. Anthropic's September 2025 postmortem described infrastructure bugs that at the worst hour affected 16% of requests to one model, and said its evaluations had not captured what users were reporting.³⁰ OpenAI rolled back a model update within four days after A/B tests, offline evaluations and expert reviews all missed a change in its behavior.³¹ If the labs' own evaluations miss regressions, a pilot's golden set will miss them too, which is why production evaluation runs on live traffic and never stops.

Agents add durable state, compounding errors and a much larger attack surface. Anthropic's account of its multi-agent research system says the last mile often becomes most of the journey, that one failed step can send an agent down a different path, and that full production tracing was how its team diagnosed failures.²⁸ EchoLeak, a zero-click prompt injection in Microsoft 365 Copilot rated 9.3 for severity, showed how one crafted email could pull internal data out through an assistant; Microsoft fixed it server-side and reported no exploitation.²⁷ OWASP's 2025 list for LLM applications names excessive agency and unbounded consumption among its top risks, and both are invisible in a demo.²⁶ Our guide to agent guardrails sets out the controls.

The bill that only appears at volume

Token prices for a fixed level of capability are collapsing, by 9 to 900 times a year depending on the task in Epoch AI's measurement.⁷ Bills rise anyway. Gartner calls this the inference paradox: cheaper models tempt teams into agentic designs that reason, retry and check their own work, and it predicts that inference cost per agentic workflow will rise more than fivefold through 2028.⁶ It also expects inference to make up at least 70% of a model's lifetime cost, and at least half of generative AI projects to overrun their budgets through 2028 because of poor architectural choices and thin operating experience.³²

The clearest public evidence came from internal developer tools. Uber opened agentic coding tools to most of its engineers in December 2025, reportedly used up its 2026 AI budget by April, and then capped spending per tool.³³ On the customer side, Gartner predicts that by 2030 the generative AI cost per customer-service resolution will exceed what many offshore human agents cost, as vendors move from subsidized growth to profit.³⁴ A pilot business case built on list token prices and chat-length tasks understates production cost three ways at once: fifty pilot users become five thousand, agent loops multiply the tokens per task, and vendor pricing is heading up.

One test ties the cost drivers together: the cost per successful task at projected volume, set against the measured cost per task today, counting staff time, system cost, rework and the expected loss from errors. Gartner warned in 2024 that CIOs who do not understand how their generative AI costs scale could miscalculate them by 500% to 1,000%, so the pilot is the place to measure both sides.³⁵ Our guides to AI ROI and inference cost show how to build both sides of that comparison.

Public failures and the control each one lacked

There are now enough public cases of AI systems withdrawn, rolled back or ruled against to show a pattern. In almost every one, the model's raw capability was a minor factor. A control was missing.

Two details matter more than the headlines. Most of these cases ended in redesign: Klarna runs AI and people side by side, and Commonwealth Bank kept its bot and reversed only the staffing decision.⁴¹⁴² And a popular retelling is wrong. Klarna's "700 agents" was a workload equivalent in its 2024 announcement, and its filing with the SEC reports the assistant handling 66% of service chats in the year to March 2025.⁴⁰⁴⁷ Quoting these cases accurately is part of being taken seriously by a board that has read them too.

What the organizations that scale do differently

Workflow redesign is the most consistent difference. In McKinsey's 2026 survey nearly three-quarters of high performers had fundamentally redesigned workflows around AI, against about a quarter of everyone else.³ Its March 2025 edition tested twelve adoption practices and found that tracking well-defined KPIs had the largest effect on the bottom line, while fewer than one organization in five did it.⁴⁸ High performers are also more likely to have defined rules for when a model's output needs human validation.¹²

BCG adds the organizational detail. Future-built companies are 1.5 times as likely to share AI ownership between business and IT, and BCG names exclusive IT ownership as a marker of stagnating firms. They also concentrate: 62% of their initiatives are deployed, against 12% at laggards that spread effort across scores of workflows.¹³ MIT's successful buyers found use cases through frontline managers, judged tools on operating outcomes instead of model benchmarks, and started at the edge of a workflow before moving inward.⁴

Uber's code-review system shows the method in practice. It rolled out one team at a time, tracked precision on dashboards, counted false positives reported by engineers, and compared itself with human reviewers. It now covers more than 90% of about 65,000 weekly code changes, and engineers rate 75% of its comments useful.⁴⁹ The figures are Uber's own, and the method is the lesson: one workflow, one usefulness metric, and expansion only when the number held.

Ten gates a pilot passes before go-live

No single framework publishes a production gate for generative AI, but the public guidance agrees on what one should contain. AWS's Generative AI Lens covers output quality, excessive agency, observability, versioning and inference cost.⁵⁰ Microsoft's Responsible AI Standard requires release criteria for each metric and error type before release, named owners who can override or stop the system, and a documented fallback for each predictable failure with the time it takes to invoke.⁵¹ NIST's Generative AI Profile sets out the measurement and management duties behind them.⁵² The table combines those sources into the gates we would hold any release to.

No vendor-neutral standard sets the numbers, such as a minimum share of answers grounded in a source. The person who owns the use case has to write them, and gate 2 is the one most pilots skip. A pilot without written thresholds can neither pass nor fail, which is exactly how a pilot ends up in evaluation for over a year.

Run the pilot the way production will run

The practical answer is a pilot that already looks like production, run in an order where each step earns the next. Accenture research published in Harvard Business Review found that firms that design for scale from the start attempt scaling, and succeed, about twice as often as firms stuck in a proof-of-concept habit.⁵⁴

1. 1Choose one workflow with an owner Pick a narrow, high-value workflow whose business owner holds the metric, and measure today's baseline before anything is built.
2. 2Write the bar first Agree the success metric and the go or no-go threshold, and sign them before the first line of code.
3. 3Build on production data Connect through the real integrations and permissions. A cleansed extract proves nothing about the system that will run.
4. 4Run in shadow mode The new system receives a copy of live requests while only the current process answers users, so quality is measured on real traffic at no risk.⁵⁵
5. 5Move to approval mode A person approves each action, and every correction becomes a new evaluation case.
6. 6Release behind a canary Send a small slice of traffic with rollback ready, widen autonomy only for low-risk actions, and keep irreversible actions with people until reliability is proven.⁵³²⁹

Autonomy is a design decision, separate from what the model can do. A 2025 paper by researchers at the University of Washington describes five levels for the user's role, from operator through collaborator, consultant and approver to observer, and the right level for each action can be set deliberately and raised with evidence.⁵⁶

Rescue, restart, buy or stop

A stalled pilot has four honest exits. Carrying on as before without new evidence is how pilots reach their second year, so it belongs on no one's list of options.

Stopping costs less politically than most teams fear: 81% of Zapier's respondents said a failed pilot had minimal to moderate impact on future AI investment.⁹ The warning sign for a stop is a plateau. Hamel Husain, who has worked on many production LLM systems, describes the pattern where fixing one failure mode exposes others, and puts error analysis and evaluation at 60 to 80% of development time in the projects he describes.⁵⁷ Weak evaluation is the most common reason a pilot can neither ship nor be killed.

The evidence on who should build is mixed and worth reading carefully. In MIT's interview sample, external partnerships using customized tools reached deployment about 67% of the time, against about 33% for internal builds, with the authors warning that the figures are self-reported and the correlation may not be causal.⁴ Menlo Ventures found the share of AI use cases bought instead of built rose from 53% in 2024 to 76% in 2025, and a16z's CIOs said internal tools are hard to maintain as models change; both firms invest in AI application companies.⁵⁸⁵⁹ Pulling the other way, 32% of McKinsey's 2026 respondents decided against buying software because they could build it with coding agents.³ And Gartner predicts that by 2028 70% of enterprises will abandon agentic AI built by a vendor's forward-deployed engineers, because of rising costs and an inability to evolve it themselves.⁸

Our reading of that evidence is simple. Buy where the workflow is generic. Bring in a partner for the production transition, and write the handover of evaluation suites, runbooks and code ownership into the contract. Build where the workflow sets you apart and your team can own evaluation and operations. Our questions for an AI development company cover how to test a partner on exactly that.

What a pilot-to-production assessment should answer

An outside assessment is worth paying for only if it ends in one of the four exits with reasons. It needs people who have run production systems: an AI engineer for prompts, agents, retrieval and tools, a data engineer for pipelines and index freshness, a platform engineer for deployment, cost routing, monitoring and model migration, and a reviewer for security and compliance. Your side keeps the business owner, a domain expert who maintains the evaluation set, and the sponsor. That split matches the skills market: ManpowerGroup's 2026 survey of 39,000 employers ranks AI model and application development as the hardest skill to hire.⁶⁰

The pilot that impressed the board did its job. It showed the idea can work. Whether it becomes a system the business runs on is decided by the next set of choices: a metric someone owns, data the system can reach in production, gates written before the release, and costs measured at the volume the business will actually see.

This is the work behind our AI development service. We take a pilot through the assessment above, write the gates with the person who owns the metric, run shadow and canary releases on production data, and hand over the evaluation suite, runbooks and code so your team owns the system when we step back.