Loading lesson…
Mathematics for AI / Calculus & Probability
Reasoning under uncertainty.
Reviewed by Yuvaraj
Every modern classifier hands you a number between 0 and 1 and calls it a probability. To read that number correctly, to know whether a model that is "99% confident" should actually be trusted, you need the small, sturdy core of probability theory. This lesson builds that core from scratch: the space of possible outcomes, the three rules every probability obeys, how one event shifts the odds of another, and the single formula (Bayes' theorem) that quietly powers spam filters, diagnostic models, and the loss functions you optimize every day.
Start with an experiment whose outcome is uncertain. The sample space is the set of all possible outcomes. For a single die roll, . An event is any subset of , for example "the roll is even" is the event . Probability is a function that assigns each event a number measuring how likely it is.
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
Every valid probability obeys three rules (Kolmogorov's axioms):
Everything else, including Bayes' theorem, is derived from these three lines.
New information changes the odds. The probability of given that occurred is
Intuitively, you shrink your world down to the outcomes where is true, then ask what fraction of that smaller world also has . Two events are independent when learning tells you nothing about , i.e. , equivalently .
Rearranging the definition both ways gives us a way to flip a conditional:
This lets us turn , which is often measurable, into , which is what we actually want.
A disease affects 1% of people. A test detects it 99% of the time when present () but also fires falsely 5% of the time in healthy people (). You test positive. What is ?
Only about 16.7%, a positive test still leaves you probably healthy, because the disease is rare. Natural frequencies make this obvious. Imagine 10,000 people:
| Group | Count | Test positive |
|---|---|---|
| Have disease (1%) | 100 | 99 |
| Healthy (99%) | 9,900 | 495 |
| Total positives | 594 |
Of the 594 positives, only 99 are truly sick: , matching Bayes exactly.
A random variable maps outcomes to numbers (a reward, a count, a loss). Its expectation is the probability-weighted average:
Suppose an agent's per-step reward is with probability , with probability , and with probability :
The agent nets per step on average, the quantity reinforcement learning tries to maximize.
Why this is everywhere in AI
Training a classifier with cross-entropy loss is minimizing an expectation over your data, and Naive Bayes classifiers apply Bayes' theorem directly. The 16.7% result is exactly why calibration matters: a confident-looking output can still be wrong when the base rate is low.
Common mistakes