When to Fine-Tune an LLM in 2026: What It Fixes, What It Costs and Why the Market Moved

Fine-tuning means training an existing language model further on your own examples so that it behaves the way you need. For three years it was the default answer when a company wanted a model of its own. In 2026 the market moved. OpenAI began winding down its fine-tuning platform, other providers deprecated or narrowed theirs, and the strongest general models now match many tuned ones with a good prompt and the right documents. Fine-tuning still pays in specific cases, and those cases are easier to identify than they were. This guide covers what fine-tuning changes, where it loses and wins against the alternatives, what companies report, what it costs on each platform, which risks and rules come with it, and how to run a project that ends in a model worth keeping.

• 9%of enterprise production models were fine-tuned in 2024, while 51% of enterprises used retrieval¹⁵
• 12.1%of comparisons in which medical fine-tuned models beat the general model they were built on²
• 31.8%harmful-output rate of GPT-3.5 Turbo after fine-tuning on a benign dataset, up from 5.5%¹⁴

What fine-tuning changes, and what it does not

The most useful finding for a buyer is that fine-tuning and retrieval solve different problems. Fine-tuning changes how a model behaves. Retrieval, usually called RAG, changes what it knows at the moment it answers by placing relevant documents in front of it. Microsoft researchers tested both on 910 questions about events after the models' training cutoff. Retrieval lifted Mistral 7B from 48.1% to 87.5%; fine-tuning on the same source text reached 50.4%, and for Llama 2 7B it cut accuracy from 35.3% to 21.9%.¹ The authors conclude that models "struggle to learn new factual information through unsupervised fine-tuning."¹⁶ A study of less popular facts found that "RAG surpasses FT by a large margin."¹⁷

Fine-tuning on facts can also make a model worse. Google researchers found that examples carrying facts the model did not already know are learned "significantly slower," and once learned "they linearly increase the model's tendency to hallucinate."¹⁸ The practical rule follows: keep changing facts in retrieval, and build a fine-tuning set from examples whose facts the base model already handles. The two can add up when the target is behavior inside a domain. In a Microsoft agriculture study, fine-tuning added more than 6 percentage points of accuracy and retrieval added 5 more on top.¹⁹

Specialized models keep getting overtaken

The history of domain models is a history of being overtaken by the next general model. BloombergGPT, a 50 billion parameter model trained on a 363 billion token financial dataset,²⁰ lost to five-shot GPT-4 on financial sentiment, with F1 scores of 0.86 against 0.51, and on conversational financial questions, 76.48 against 43.41.²¹ In medicine, GPT-4 with a prompting method called Medprompt outperformed the specialist Med-PaLM 2 "with an order of magnitude fewer calls" and passed 90% on the MedQA exam for the first time.³ A year later, o1-preview scored 96.0%, and few-shot prompting made it worse.²²

A peer-reviewed audit of medical models built from general ones found that the medical versions beat their own base models in only 12.1% of three-shot comparisons, tied in 49.8% and were significantly worse in 38.2%.² An adaptation tied to one base model has a shelf life measured in model generations, and a prompting recipe tuned to one generation can hurt the next.

Where fine-tuning still wins

The same financial study that embarrassed BloombergGPT shows where tuning keeps winning. On headline classification, a fine-tuned BERT model scored 95.36 against 86.00 for five-shot GPT-4.²¹ A 2024 study found fine-tuned small models "(still) significantly outperform" zero-shot generative models at text classification.²³ In 2026, a fine-tuned Qwen2.5-0.5B reached a micro-F1 of 0.83 on relation extraction, against 0.69 for GPT-5.4 and 0.66 for Claude Sonnet 4.6 used zero-shot.⁴ Distillation compounds the effect: a 770 million parameter model trained on the reasoning of a 540 billion parameter model outperformed it on a benchmark using only 80% of the labeled data.²⁴

The most quoted result in this area needs its conditions attached. LoRA Land fine-tuned 310 small models across 31 tasks and found they beat GPT-4 by 10 points on average, with 224 of 310 above GPT-4's score.²⁵ GPT-4 was queried with "zero or single-shot, completion-style prompts," and the paper came from a company that sold fine-tuned model serving. Behavior also needs far less data than knowledge. LIMA tuned a 65 billion parameter model on 1,000 curated examples and was judged equal to or better than GPT-4 in 43% of comparisons.²⁶ OpenAI's guidance is to start with 50 good examples and, "if 50 examples have no impact, rethink your task or prompt before adding training data."²⁷

Long context is the other alternative. A Google DeepMind study found that when resourced, putting the whole text in the model's context "consistently outperforms RAG" on average, while retrieval costs far less, and that routing each question to one or the other kept quality close to long context at much lower cost.²⁸ For a document set that fits in the window and is reused, cached long context removes much of the case for baking knowledge into weights.

What companies report

The production cases with numbers share one shape: a narrow, high-volume task moved from a frontier model to a small fine-tuned one, where the gain is mostly cost and speed. Checkr runs more than 1.5 million background checks; its engineer reported that a fine-tuned small model reached 97% accuracy against 88% for GPT-4, answered in about half a second, and cost about $800 a month against an estimated $7,000 to $12,000 with GPT-4.⁵ LinkedIn's domain-adapted Llama 3.1 8B models, trained on about 200 million tokens, were "75x and 6x cost effective" compared with GPT-4 and GPT-4o.⁶ Shopify runs 40 million multimodal inferences a day for its product catalog and cut median latency from 2 seconds to 500 milliseconds.²⁹ A study that replaced a production GPT-4 feature with small open models measured a cost reduction of 5 to 29 times.³⁰

Reinforcement fine-tuning, which trains a model against a grader that scores its answers, is the newer route, and its published results come from the seller. OpenAI reports that Ambience's medical coding model rose from 0.39 to 0.57, above a physician baseline of 0.45, and that Harvey's legal extraction F1 rose from 0.563 to 0.6765.³¹ The 2026 headline cases are larger projects on open weights the companies control. Harvey's Tenet, post-trained on the open Kimi K3 model, completes "almost twice as many held out tasks" as its base on Harvey's benchmark, at "roughly one-tenth the cost per cell" on one workload.³² Thomson Reuters spent $40 million training its own model from an open foundation.³³ Both are company evaluations, and both are far from a typical project.

Adoption data shows fine-tuning is a minority practice. In Menlo Ventures' 2024 survey of 600 enterprise leaders, retrieval reached 51% adoption while "only 9% of production models" were fine-tuned.¹⁵ Andreessen Horowitz's 2025 survey of chief information officers reported "Fine-tuning viewed as less necessary as model capabilities improve," quoting one enterprise: "you just dump it into a long context and get almost equivalent results."³⁴

Managed fine-tuning retreated in 2026

OpenAI's documentation now opens its fine-tuning guides with "OpenAI is winding down the fine-tuning platform."²⁷ The company has published no detailed reason. Its own model optimization guide says prompt engineering "may be all you need,"³⁸ and the evidence above suggests why customers drifted away: for most tasks, a stronger general model with a better prompt and the right documents caught up with the tuned model, and a tuned model has to be rebuilt each time its base is retired.

What it costs

At these rates, tuning on a million tokens for three epochs costs a few dollars on an open model and about $30 on Gemini 3.5 Flash. Training on your own hardware is in the same range: QLoRA showed in 2023 that a 65 billion parameter model can be fine-tuned "on a single 48GB GPU,"⁴³ so a day of single-GPU training costs about $100 at Lambda's list price.⁴¹ The recurring bill is where platforms differ. Per-token billing at the base price makes a low-traffic tune almost free to keep. A 1.5 times markup scales with traffic. Azure's $1.70 hourly fee is about $1,200 a month per deployed model before any tokens.¹² For teams running their own models, S-LoRA serves thousands of fine-tuned adapters with "up to 4 times" the throughput of naive serving, so one tune per task or per customer does not mean one GPU each.⁴² The larger costs are the data, the evaluation work and the retrain at every base-model change.

Risks that move into the weights

Fine-tuning can undo a provider's safety training. Researchers broke GPT-3.5 Turbo's guardrails by fine-tuning it on "only 10 such examples at a cost of less than $0.20," raising its harmful-output rate from 1.8% to 88.8%. The more important result for a company is the accidental one: one round of tuning on Alpaca, a widely used benign dataset, raised the rate from 5.5% to 31.8%.¹⁴ Narrow tuning can spread further than intended. Models fine-tuned to write insecure code without saying so began asserting "that humans should be enslaved by AI" on unrelated prompts, and a trigger could hide the behavior until it appeared.⁴⁴

Training data can leak back out. A 2026 study showed that "small, proprietary SFT datasets can induce meaningful privacy leakage" of personal information in medical and legal settings.⁴⁵ Replacing real records with generated data is no cure: after tuning on generated data, successful extractions of personal data rose by over 20% in one model family.⁴⁶ Tuning also costs general ability. A study of continual fine-tuning found forgetting across models from 1 billion to 7 billion parameters, worsening as size grew,⁴⁷ and LoRA, the common low-cost method, "substantially underperforms full finetuning" while better preserving skills outside the target task.⁴⁸

A hosted fine-tune also has an end date. None of the closed platforms we reviewed documents a way to export tuned weights. Azure publishes retirement dates for fine-tuned deployments, such as October 14, 2027 for GPT-4.1,¹³ and Bedrock restricts customization once a base model turns Legacy.³⁶ The lasting assets are the training data, the evaluation set and the recipe, so plan to keep and version them.

Licenses and rules

How to run a fine-tuning project that pays

Every provider gives the same first instruction: "Good evals first! Only invest in fine-tuning after setting up evals."²⁷ A fine-tuning project is an evaluation project with a training step. The tuned model must beat three baselines: its own base model, the strongest current model with a good prompt and retrieval, and the next model release when it arrives. If it cannot beat the second, stop before training.

1. 1Write the evaluation first Build a held-out set from real logged or expert-labeled cases, add a safety and general-ability check, and freeze it.
2. 2Set the baselines Score the strongest current model with a careful prompt, retrieval or long context for knowledge, and a smaller model for cost.
3. 3Name the gap Go ahead only if the remaining gap is about behavior, format, classification, or cost and latency at volume.
4. 4Clear the ground Record the base model's license, the rights to each training source and the lawful basis for any personal data.
5. 5Pilot with 50 examples Curate 50 to a few hundred examples, filter out facts the base model does not know, and remove personal data you do not need.
6. 6Train small and test hard Start with LoRA on an open model, then rerun the task, safety and general-ability checks and an extraction test before release.
7. 7Plan the retrain Version the data, evaluation and recipe, rerun the comparison at every major model release, and retire the tune when a prompted model matches it.

The question has changed from "should we fine-tune GPT?" to "is this task narrow, high-volume and measurable enough to justify owning a small model?" When the answer is yes, the economics are better than ever, with training at tens of dollars, adapters that share hardware and permissive licenses. When it is no, a strong general model with retrieval will usually match the tuned model now and beat it after the next release. Either way, the evaluation set and the curated data are the assets that last. The weights can be rebuilt.

This is how we approach model customization in our AI development work: build the evaluation first, prove what prompting and retrieval achieve, and fine-tune a small open model only where the numbers show a gap worth owning, with the data, tests and recipe kept so it can be rebuilt on the next base model.