Loading lesson…
Fine-Tuning / When & How
And when prompting or RAG is the better answer.
Reviewed by Yuvaraj
Fine-tuning sounds like the obvious way to make a model "yours," but most of the time it is the wrong first move. The single most useful thing to internalize is what fine-tuning actually changes: it updates the model's weights to bake in a behavior, a format, a tone, a style, a decision boundary, that you would otherwise re-specify in every prompt. It is very good at teaching a model how to respond. It is unreliable at teaching a model what is true right now. So if your real problem is "the model doesn't know our latest facts," fine-tuning is almost never the answer, that is what RAG is for.
Every response depends on two things: the model's weights (its trained parameters) and the context you pass in (your prompt, examples, and any retrieved text). You can move either lever.
Fine-tuning is the second lever. That is exactly why it is powerful for behavior and a poor fit for knowledge that keeps moving.
For a curious beginner
Changing the prompt is like leaving a sticky note for the model each time, it reads your note, then forgets it the moment the request ends. Fine-tuning is like sending the model on a training course: it comes back with new habits it applies automatically, no note required.
How it is actually used
The prompt lives in the input tokens at inference time; nothing about the model changes, so behavior is editable per request and instantly reversible. Fine-tuning runs gradient updates over your examples and writes new values into the weights (or into small adapter matrices with LoRA/QLoRA), so the behavior persists across every future call with zero prompt overhead, at the cost of a training run and an eval.
The underlying mechanism
A forward pass computes , where is your context and the weights. Prompting changes ; fine-tuning changes by descending a loss over your dataset, . LoRA freezes and learns a low-rank update so the effective weights become , with far smaller than .
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
| The ask | Best approach | Why |
|---|---|---|
| "Answer questions about our constantly-updated policy docs" | RAG | The knowledge changes weekly; retrieve it at query time instead of freezing it into weights. |
| "Always output valid JSON in our fixed schema" | Fine-tune (after prompting falls short) | A stable, repeatable format is a behavior, bake it in and stop paying for schema instructions on every call. |
| "Adopt our brand voice in every reply" | Prompt first, then fine-tune | A system prompt usually nails tone; fine-tune only if you need it rock-consistent at scale without the prompt overhead. |
| "Classify support tickets into 12 categories cheaply" | Fine-tune a small model | A narrow, high-volume task is where a fine-tuned small model matches a big one at a fraction of the cost and latency. |
Reach for it once a good prompt and, where relevant, RAG are in place and you still have a real, repeatable gap: a behavior you need identically every time, a prompt so long that latency or token cost hurts, or a small, cheap model you want to lift to a bigger model's quality on one narrow task. In all three, you are encoding behavior, not facts. And you only know it worked if you measured it, a held-out evaluation set is not optional.
Common mistakes