What is fine-tuning? How to specialize an AI model

You have three ways to make a general-purpose AI model do what you specifically want. You can write a better prompt. You can give it access to your documents through retrieval. Or you can fine-tune it: continue its training on your own examples until the behaviour you want is baked into the model itself. Fine-tuning is the most powerful of the three and also the most misunderstood, the tool teams reach for first when they should reach for it last.

This piece explains fine-tuning in full: what it actually does to a model, why the distinction between changing behaviour and adding knowledge determines when it helps, the breakthrough that took it from a research-lab expense to something you can run on a gaming GPU over a weekend, and the honest answer to the question everyone asks too early, "should we fine-tune?" By the end, you should be able to tell whether fine-tuning is the right tool for a given problem or an expensive detour, which is most of the value in understanding it at all.

The one distinction that matters: form, not facts

Start with the idea that governs everything else, because getting it wrong is the most common and costly fine-tuning mistake. Fine-tuning is for form, not facts. It is the right tool for shaping how a model behaves, and the wrong tool for injecting what it knows.

Here is the distinction. If you want a model to always respond in your brand's voice, produce output in a rigid JSON schema, follow a specialised workflow, or reliably refuse certain requests, that is behaviour, a durable pattern you want repeated every time, and fine-tuning is well suited to it. If instead you want the model to know your company's current pricing, this quarter's policies, or any information that changes, that is knowledge, and fine-tuning is the wrong tool: it is expensive to update, it bakes the information into weights that go stale, and it tends to blur specifics rather than store them exactly. Knowledge that changes belongs in retrieval, where you can update a document and see the answer change instantly, as covered in the piece on RAG versus fine-tuning. Behaviour that should stay fixed belongs in the weights.

Almost every fine-tuning regret traces back to crossing this line: teams fine-tune to teach the model facts, watch those facts go out of date, and pay to retrain. Keep form and facts separate and most of the confusion dissolves.

The escalation ladder: most teams should not fine-tune

The honest framing that experienced teams converged on in 2026 is a ladder, tried in order: prompt, then retrieve, then fine-tune, then distill. Each rung is more powerful and more expensive than the last, and you climb only when the one below actually fails.

Start with prompting, because it is free, instant, and reversible: a better instruction or a few examples in the prompt solves a surprising share of problems. If the model needs information it does not have, add retrieval before touching the weights. Only when prompting and retrieval both fail to give consistent results, and you can point to a specific, measurable gap, does fine-tuning earn its place. The uncomfortable truth most practitioners will tell you is that the question "should we fine-tune?" almost always arrives before the prerequisite work is done, and the honest answer is usually "not yet." Fine-tuning is a real tool with real uses, but it is the third thing to try, not the first, and cheaper tools solve most problems.

What fine-tuning actually does

When fine-tuning is the right call, here is the mechanism. A large language model arrives already trained on an enormous amount of text (its pretraining), which gave it broad capability. Fine-tuning takes that finished model and continues its training, but now on a small, curated set of examples that demonstrate exactly the behaviour you want: pairs of input and ideal output, in your format, your tone, your task. The model adjusts its internal weights to reproduce that behaviour, and because the capability was already there from pretraining, it takes remarkably few examples (thousands, sometimes hundreds) to shift the behaviour, not the millions that pretraining required.

The contrast with retrieval is worth stating precisely, because it is the crux. Retrieval changes the model's input, adding relevant text to the prompt at the moment you ask. Fine-tuning changes the model's weights, altering the model itself so the behaviour persists without anything added to the prompt. One adjusts what the model sees; the other adjusts what the model is. That is why fine-tuning produces durable, consistent behaviour and lower latency (no retrieved text to process), and also why it cannot cheaply keep up with changing facts.

The problem with full fine-tuning

The original way to fine-tune was to update all of the model's parameters, and in 2026 this "full fine-tuning" is almost never the right choice for a product team. Three problems make it painful. It is expensive: updating every one of billions of weights takes serious compute and memory. It risks catastrophic forgetting, where teaching the model your narrow task degrades the general capability it had before, since the same weights encode both. And it locks you to one base-model checkpoint: your fine-tune is a whole new multi-billion-parameter model, costly to store and impossible to move to a better base without redoing the work.

For years these costs meant fine-tuning was the preserve of well-resourced labs. Then a single idea changed who could do it.

The breakthrough: LoRA and parameter-efficient fine-tuning

The technique that democratised fine-tuning is called LoRA (Low-Rank Adaptation), and it belongs to a family known as parameter-efficient fine-tuning, or PEFT. The insight behind it is worth understanding, because it is both simple and a little surprising.

The observation is that the change fine-tuning makes to a model, though it touches a huge weight matrix, actually has a low "intrinsic rank": the useful adjustment lives in a much smaller space than the full parameter count suggests. So instead of updating the giant weight matrices directly, LoRA freezes the entire base model and inserts small, trainable adapter matrices into its layers (specifically into the attention and related layers of the transformer). Only those tiny adapters are trained, typically around 0.1 to 1 percent of the original parameter count. The frozen base does the heavy lifting; the small adapters steer it toward your task.

This buys three things at once. Cost collapses, because you are training roughly 1 percent of the parameters instead of 100 percent, reaching quality comparable to full fine-tuning at a fraction of the compute. Forgetting is limited, because the base model's weights are frozen and its general capability stays intact. And the result is portable: your fine-tune is now a lightweight adapter file, a few megabytes, that you can swap in and out on top of the base, even keeping several task-specific adapters and loading whichever you need. LoRA turned fine-tuning from producing a whole new model into producing a small clip-on module.

QLoRA pushed this further by attacking memory. It loads the frozen base model in 4-bit precision (a form of quantization that shrinks each weight to a quarter of its usual size) while training the LoRA adapters in higher precision on top. The fidelity cost is small and the effect is dramatic: models that used to require multiple high-end datacentre GPUs became fine-tunable on a single consumer graphics card. This is the real democratisation. A task that once needed a research budget can now be done on hardware a hobbyist might already own, which is why fine-tuning went from rare to routine.

When fine-tuning is the right tool

Having stressed restraint, it is worth being concrete about the cases where fine-tuning clearly wins, because they are real. Reach for it when you need rigid, repeated behaviour at scale that prompting cannot make reliable: a consistent brand voice across every output, strict adherence to a structured format like a legal template or a specific JSON schema, or a specialised task the base model performs inconsistently. Reach for it when you need lower latency or cost, since a fine-tuned smaller model can match a prompted larger one on a narrow task while being cheaper to run. And reach for it for distillation: fine-tuning a small model on a large model's outputs to capture much of the big model's skill in a cheaper package. In each case the pattern is the same: a well-defined behaviour, needed reliably and repeatedly, that prompting and retrieval could not pin down.

The precondition that decides success, more than the method, is data quality. A fine-tune is only as good as its examples: clean, consistent, correctly formatted demonstrations of the target behaviour, enough of them to be representative, with a clear metric that proves the result actually improved. Most failed fine-tunes fail not on the algorithm but on the dataset. If you cannot describe the exact behaviour you want in a few hundred clean examples and state how you will measure success, you are not ready to fine-tune yet.

The short version

Fine-tuning adapts a general model to your task by continuing its training on your examples, changing the model's weights so the behaviour persists. Its governing rule is that it shapes form, not facts: use it for durable behaviour like voice, format, and specialised tasks, and use retrieval instead for knowledge that changes. It sits third on the ladder after prompting and RAG, and most teams reach for it too early. Full fine-tuning is expensive and risks catastrophic forgetting, so the modern default is LoRA, which freezes the base model and trains tiny low-rank adapters at around 1 percent of the cost, with QLoRA's 4-bit quantization making even large models trainable on a single consumer GPU. Throughout, data quality decides the outcome.

The idea to hold onto is that *fine-tuning changes what a model is rather than what it sees, which makes it the right tool for durable behaviour and the wrong tool for changing facts, and thanks to LoRA it is now cheap enough that the real question is not "can we?" but "should we, for this?"* The power was never in doing it. It is in knowing when it beats the simpler tools, and when it does not.

Common questions

What is fine-tuning in AI? Fine-tuning is the process of taking a pre-trained, general-purpose model and continuing its training on a smaller, curated set of examples so it adapts to a specific task, style, or behaviour. It changes the model's internal weights, so the new behaviour becomes part of the model itself rather than something added at the prompt. Because the model already learned broad capability during pretraining, fine-tuning needs relatively few examples to shift its behaviour, often thousands rather than the trillions of tokens used in pretraining.

What is the difference between fine-tuning and RAG? They change different things. Fine-tuning changes the model's weights to durably alter its behaviour, which is best for form: tone, format, and specialised tasks. RAG (retrieval-augmented generation) changes the model's input by adding relevant documents at query time, which is best for facts: knowledge that is proprietary or changes frequently. The rule of thumb is form versus facts. They are complementary and often combined: a fine-tuned model for consistent behaviour, paired with retrieval for up-to-date knowledge.

Should I fine-tune or just use prompting? Usually start with prompting. The sensible escalation ladder is prompt, then RAG, then fine-tune, then distill, tried in that order because each is more powerful and expensive than the last. Prompting is free, instant, and reversible and solves a surprising share of problems. Fine-tune only after prompting and retrieval have both failed to give consistent results and you can point to a specific, measurable gap. Most teams asking whether to fine-tune have not yet done the cheaper prerequisite work, and the honest answer is often "not yet."

What is LoRA? LoRA (Low-Rank Adaptation) is the dominant parameter-efficient fine-tuning method. Instead of updating all of a model's billions of parameters, it freezes the entire base model and inserts small, trainable adapter matrices into its layers, training only those, typically around 0.1 to 1 percent of the original parameters. It works because the useful change fine-tuning makes has a low intrinsic rank, so small matrices can capture it. LoRA slashes cost, limits catastrophic forgetting by keeping the base frozen, and produces a lightweight, swappable adapter file rather than a whole new model.

What is the difference between LoRA and QLoRA? Both freeze the base model and train small low-rank adapters. QLoRA adds one thing: it loads the frozen base model in 4-bit precision (quantization) while training the adapters in higher precision on top. This dramatically cuts memory use with only a small fidelity trade-off, which is what made fine-tuning large models possible on a single consumer GPU rather than a cluster of datacentre cards. In short, LoRA reduces how many parameters you train; QLoRA also reduces the memory the frozen base consumes.

What is catastrophic forgetting? Catastrophic forgetting is when fine-tuning a model on a narrow new task degrades the general capabilities it had before, because the same weights encode both the old and new abilities and training on the new task overwrites some of the old. It is a major risk of full fine-tuning, which updates every weight. Parameter-efficient methods like LoRA reduce it substantially by freezing the base model's weights and training only small added adapters, so the original capability stays largely intact.

How much data do I need to fine-tune a model? Less than people expect, but quality matters far more than quantity. Because the base model already has broad capability from pretraining, fine-tuning often needs only hundreds to thousands of clean, consistent, correctly formatted examples that demonstrate the exact behaviour you want. The common failure is not too little data but poor data: inconsistent formatting, mixed quality, or examples that do not clearly show the target behaviour. Just as important is defining a metric up front that proves the fine-tune actually improved the outcome.