Running AI in Production: How to Operate AI Systems After Go-Live

AI budgets are written for the build. The launch gets a date and a demo, and then the system meets what no pilot tests: time. In the months after go-live the provider updates the model, announces its retirement and has an outage or two, the users ask things nobody put in the test set, and the token bill grows with every new feature. The AI Incident Database recorded 362 incidents in 2025, up from 233 the year before.⁹ This playbook covers how AI systems fail once they are live, who notices, and the operating model that keeps them reliable: what to measure, how to manage model changes, what the rules now require and who should own the work.

• 30 daysa serving bug degraded a frontier model's answers before user reports exposed it; the provider's own evaluations were too noisy to catch it¹
• 60 daysminimum notice Anthropic gives before retiring a model; seven of its model IDs were retired in the first nine months of 2026²
• 37%of teams building agents evaluate live traffic, against 52% who evaluate offline before release⁵

How AI systems fail after go-live

The public record of failures after launch falls into a few patterns. The provider changes the model or the serving stack underneath the product, or goes down. A customer-facing assistant states a policy or a law that does not exist, and the deployer is held to it. Users talk a bot into commitments nobody meant it to make. An agent with production access destroys data, or a leaked key runs up a bill. The table lists cases from each pattern, how each came to light and the control that would have caught it sooner.

Read the third column. Operators' monitoring spotted the hard outages. Almost everything else, including wrong answers, harmful answers and a model quietly getting worse, was found by customers, reporters or the invoice. Anthropic's postmortem is candid about why: its evaluations were too noisy to separate the broken serving paths from the working ones, and privacy controls limited what engineers could see of the failing conversations. It has since committed to running its evaluations continuously on production systems.¹

When the assistant is wrong, the deployer pays. The tribunal in the Air Canada case refused the argument that the chatbot was responsible for its own words.¹³ Klarna, which had moved much of its customer service to an AI assistant, said in 2025 that cost had been a too predominant factor and that the result was lower quality, and began recruiting people again.¹⁹ A system that answers customers has to be held to a quality standard as firmly as to a cost one.

The model under you changes, then retires

A production AI system usually runs on a model somebody else owns, and the owner changes it on its own schedule. Every major provider now publishes a lifecycle policy. The notice periods leave little slack for evaluating a replacement, testing it and releasing it through change control.

Retirement is routine. Anthropic retired seven model IDs in the first nine months of 2026.² The same model can also retire on different dates in different places: Claude Sonnet 4 left Anthropic's own API on June 15, 2026, and stays on Amazon Bedrock until October 14, 2026.²²² Microsoft tells customers to start evaluating newer models against their own prompts and data without waiting for an official replacement, which it names only about 90 to 120 days before retirement.³ The working assumption for planning is a life of roughly 12 to 18 months per model version, with at least one forced migration in the life of any system.

Pinning a dated version protects against only part of the change. In 2023, Stanford and Berkeley researchers found GPT-4's accuracy at identifying prime numbers fell from 84% to 51% between March and June, and the share of its code that ran as returned fell from 52% to 10%; they named weaker instruction-following as a common factor behind the drift, and instruction-following is what an automated pipeline depends on most.²³ Anthropic's 2025 bugs changed answers with no change of model version at all.¹ An August 2026 study of 16 providers found none that let an outside party verify that the model being served is the one its documentation describes.⁴ The defence is a fixed set of your own cases run against production on a schedule, with an alert on any shift, and every replacement model evaluated on the day it ships.

Availability needs the same realism. Amazon Bedrock's agreement commits to 99.9% monthly uptime in each region, with service credits of 10% to 100% of the month's bill, and excludes downtime caused by an inoperable model.²⁴ A 99.9% month still allows about 43 minutes of downtime, and OpenAI's June 2025 incident lasted about 15.5 hours.¹² A credit is a share of the model bill, which is usually small next to what an outage costs the business, so production systems need a tested fallback to a second model or region.

Hold quality to an objective, the way uptime is

An AI service can answer every request quickly and cheaply with zero errors and still be wrong. Grafana Labs makes that point and proposes the fix site reliability teams will recognize: define an indicator for each behaviour that matters, such as grounded answers or completed tasks, score it with an evaluator on sampled conversations, and give each its own objective, for example 95% fulfilled over 30 days.²⁵ Google's SRE workbook already defines an indicator as good events over total events, allows correctness as the good event, and includes error-budget policies that freeze changes until a service is back within its objective.²⁶

The cloud providers' guidance converges on the same layers. AWS rates missing foundation-model monitoring a high risk and lists invocations, latency, token usage, errors and throttling, with incident playbooks practised for when the alarms fire.²⁷ Microsoft adds continuous evaluation of production traffic at a sampled rate, scheduled re-runs of test sets to detect drift and scheduled red teaming.²⁸ Practice lags the guidance: of the more than 1,300 practitioners LangChain surveyed, 52.4% run offline evaluations and 37.3% evaluate live traffic.⁵ Our guide to LLM evaluation covers building the test set and the release gate.

Datadog's data from its customers shows where everyday failures sit. In February 2026, 5% of model calls returned an error and 60% of those errors were rate limits, so capacity handling, backoff and fallback routing come first.²⁹ The same data shows system prompts making up 69% of input tokens while only 28% of calls on models that support caching read any cached input, which is spend that tuning can recover.²⁹ Datadog sells observability, and these figures come from its own customer base.

Instrument with OpenTelemetry so traces outlive any one vendor's dashboard. Its conventions for model calls, agents and tool calls are still in development, and in June 2026 they moved out of the core specification into a repository of their own.³⁰ Pin the version you emit and keep a collector that can remap names when they change.

Limits that hold at machine speed

Agents and keys fail faster than people react. The $82,314.44 Gemini bill accrued over two days.¹⁸ The DataTalks.Club database went in one command, and its founder wrote afterwards that delegating the plan, apply and destroy steps had removed the last safety layer. He then enabled deletion protection, daily automated restore tests and backups kept apart from the infrastructure tooling.¹⁷

The controls are ordinary, and they have to be hard limits rather than dashboards: spend caps per key and per task, least-privilege credentials, approval before any irreversible action, and an iteration limit on every agent loop, a stopping condition Anthropic's own agent guidance recommends.³¹ The switch that turns a feature off must work without the system it stops. In OpenAI's December 2024 outage, the fix depended on a control plane the failure had made unreachable.¹¹ Our AI agent guardrails playbook covers these controls in depth.

What the rules now expect after launch

Regulation has caught up with operations. The EU AI Act requires deployers of high-risk systems to use them as instructed, assign oversight to people with the competence and authority to act, monitor the system, suspend use when they suspect a risk and keep its logs for at least six months.⁶ Providers must run a documented post-market monitoring system and report serious incidents, and they may not alter the system in a way that could affect the investigation before telling the authorities, which an incident runbook has to allow for.³²⁷ The Digital Omnibus, in force since July 27, 2026, moved the high-risk dates to December 2, 2027 for Annex III systems and August 2, 2028 for Annex I, and left the incident deadlines as they were.³³

The direction is the same in every regime: an inventory, logs, monitoring, a person who can override, an incident path and accountability for vendor models. A provider's retirement schedule becomes the deployer's compliance problem, and a model swap becomes a change to assess as well as to test. Our EU AI Act guide covers classification and the obligations in detail.

Running costs more than building

Gartner expects inference to account for at least 70% of a model's lifetime cost, warns that a production-ready system can cost orders of magnitude more than the pilot, and predicts that at least half of generative AI projects will overrun their budgets by 2028 through poor architectural choices and a lack of operational know-how.⁸ None of that is new to machine learning. Google's engineers wrote in 2015 that real-world ML systems commonly incur massive ongoing maintenance costs, and that the model code is a small part of the system around it.³⁹ Our guides to inference cost and moving a pilot to production cover the economics in more depth.

The skills are scarce. Deloitte's 2026 survey of 3,235 leaders found talent the least prepared area for AI, with 20% calling their organization highly prepared, and one AI leader it interviewed discovered there was no clear inventory of the models running in production.⁴⁰ ManpowerGroup's survey of 39,063 employers ranks AI model and application development as the hardest skill to hire.⁴¹ Gartner puts 2026 spending on AI services at $576.5 billion, ahead of AI software at $461.6 billion.⁴² Incident response is maturing too: OWASP published a generative AI incident response guide in July 2025 and the Coalition for Secure AI a framework in November 2025, both written for security teams.⁴³⁴⁴ Quality incidents, such as a slow slide in answer accuracy, still need product-level owners and runbooks.

An operating model that finds problems first

The practices that close the gaps in the incident table are known and unglamorous. They work in this order, because each one depends on the one before it.

1. 1Inventory what is running Every model and version, prompt, tool, agent and key in production, each with an owner and the retirement date of its model, read from the providers' lifecycle data rather than from memory.
2. 2Trace every request OpenTelemetry traces across model calls, tool calls and agent steps, with tokens and cost per task, pinned to a known convention version and routed through a collector you control.
3. 3Set objectives for quality as well as uptime A test set built from your own cases, sampled evaluation of live traffic, an objective per behaviour, and an error-budget policy that freezes prompt and model changes when quality falls below target.
4. 4Put limits where machines outrun people Spend caps per key and per task, iteration limits on agents, least-privilege credentials, approval for irreversible actions, and a kill switch that works when the system it stops is down.
5. 5Rehearse the bad day Runbooks for a provider outage, a quality regression, a harmful answer and a cost spike, each with its fallback or rollback tested, and an incident path that meets the regulatory clocks you are under.
6. 6Treat every model change as a release Evaluate candidate models the day they ship, roll out by canary with a pinned rollback, and repeat the risk assessment before the provider's retirement date forces the move.

Launch is the point where an AI system starts depending on things its builders do not control: a provider's release and retirement calendar, the questions real users ask, a metered bill and a regulator's clock. The organizations that keep their AI reliable treat it like any production service with a few extra dials, and they decide before launch who watches those dials at three in the morning.

This is the work of our managed services. Our engineers run AI systems in production against agreed objectives for quality, latency and cost: every model version tracked to its retirement date, live traffic evaluated, spend and agent actions capped, on-call with tested runbooks, and a monthly report that puts quality beside uptime.