Loading lesson…
Machine Learning / Learning Paradigms
Predicting numbers versus predicting categories.
Reviewed by Yuvaraj
Regression and classification are the two workhorses of supervised learning: you show a model labeled examples, and it learns a function that maps inputs to outputs. The difference is simply what kind of output you want. Regression predicts a continuous number, a price, a temperature, an exam score. Classification predicts a category, spam or not, pass or fail, which of ten digits. It is the same recipe with a different target, and that one choice changes the model's final step, its loss function, and how you measure success.
Linear regression is the simplest and most important starting point. For a single input feature , it models the output as a straight line:
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
Here is the weight (the slope) and is the bias (the intercept). Training means choosing and so the line passes as close as possible to the data. "Close" is made precise by the loss function, and for regression the standard choice is mean squared error (MSE):
Squaring punishes large misses far more than small ones and keeps the loss smooth, so optimizers like gradient descent can minimize it reliably. With many features, becomes a dot product , but the idea is identical.
Worked example. Suppose we predict an exam score from hours studied, and training has settled on and . A student who studies hours gets:
Logistic regression handles the two-class case. It begins with the same linear score,
but this raw number can range from to , which is not a probability. So we squash it through the sigmoid function:
The sigmoid maps any real number into the open interval between 0 and 1, giving the estimated probability . To turn that probability into a hard label we apply a decision threshold, usually 0.5: predict class 1 when , otherwise class 0. (The natural loss here is cross-entropy, not MSE.)
Worked example. Predict the probability a student passes, using , , and hours:
Since , we classify the student as pass, with roughly 88% confidence.
import math
# Linear regression: predict exam score from hours studied
w, b, x = 8, 40, 5
y_hat = w * x + b # 80.0
# Logistic regression: probability of passing
w, b, x = 0.8, -2, 5
z = w * x + b # 2.0
p = 1 / (1 + math.exp(-z)) # 0.8808...
label = 1 if p >= 0.5 else 0 # 1 -> "pass"
The decision boundary is the surface where the model flips its prediction from one class to the other. With a 0.5 threshold, logistic regression flips exactly when , which happens at , that is, where . In our example that point is hours: study more than 2.5 hours and the model predicts pass. Because the boundary is defined by a linear equation, plain logistic regression can only draw straight-line (linear) boundaries; curved boundaries require extra features or a more expressive model.
Common mistakes