Loading lesson…
AI Foundations / How AI Learns
Two very different phases.
Reviewed by Yuvaraj
There are two completely different moments in a model's life: when it learns, and when it works. Confusing them causes a surprising amount of confusion about cost, speed, and behavior.
Training is the process from the last lessons: showing the model many examples and adjusting its parameters to reduce error. Training is:
Inference is when the trained model takes a new input and produces an output, the spam filter judging one email, the LLM answering one prompt. Inference is:
Why chatbots seem to 'learn' in a conversation
A chatbot appears to remember what you just said because your earlier messages are fed back in as part of the input (the context). The model's parameters are not changing, it is reading the conversation each turn. Close the chat and that apparent memory is gone unless the app stored it separately.
Answer from memory before revealing, retrieval practice is what builds durable recall.
During inference, a trained model:
Training a large model generally costs far more compute than a single inference call.
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
A model moves through these stages in order. Only one of them repeats:
The first four stages happen once per model version. The last runs forever, every user request is another inference on the same frozen parameters. To change what the model knows, you loop back to the top and produce a new version.
| Training | Inference | |
|---|---|---|
| What happens | Parameters are adjusted | Parameters are used, unchanged |
| Frequency | Occasional | Every request |
| Cost profile | High, upfront | Lower per use, but adds up at scale |
| Needs labels? | Usually | No |
| Result | A trained model | A single prediction |
Round numbers make the trade-off concrete. Suppose training a model costs 100,000 USD in compute, paid once. Serving it costs a small fraction of a cent per request, say 0.0002 USD.
The crossover is the key insight: for any system with real usage, inference is the cost that scales. This is why so much AI engineering effort goes into making inference cheaper (quantization, batching, smaller models) rather than making training cheaper, a later course covers exactly those techniques.
A deployed model does not learn from traffic on its own
A common misconception is that a model in production automatically gets smarter as more people use it. It does not. Inference leaves the parameters untouched. Any improvement requires a deliberate cycle: collect new data, retrain or fine-tune, evaluate, and deploy a new version. "The model will just learn from users" is not a plan, the retraining loop is.