Supervised vs Unsupervised

$\theta^* = \arg\min_\theta \; \frac{1}{n}\sum_{i=1}^{n} L\big(f_\theta(x_i),\, y_i\big)$

The paradigms differ in what plays the role of $y_i$:

• Supervised: $y_i$ is a known label you supply. The loss compares the prediction to the truth (e.g. squared error, cross-entropy).
• Unsupervised: there is no $y_i$. The loss instead scores a property of the data itself, how tight clusters are, or how well a compressed representation reconstructs the original.

Supervised learning: learn from labeled examples

You give the model input–output pairs $(x_i, y_i)$ and it learns the mapping. Two flavors:

• Classification, the label is a category. Is this email spam or not? Which digit is this?
• Regression, the label is a number. What will this house sell for? How many units ship next week?

Unsupervised learning: find structure in unlabeled data

No labels, the model discovers patterns on its own:

• Clustering, group similar records (k-means, DBSCAN). Which customers behave alike?
• Dimensionality reduction, compress many features into a few informative ones (PCA, t-SNE) for visualization or as a preprocessing step.

Worked example: one problem, framed both ways

Take a pile of emails and frame it two different ways.

1. 1Supervised framingYou have 10,000 emails each hand-labeled spam or not-spam. Train a classifier on (email text, label) pairs; it predicts the label for new, unseen emails. Success = accuracy on a held-out test set.
2. 2Unsupervised framingYou have emails with no labels. Cluster them by content similarity. The model surfaces groups, newsletters, receipts, personal mail, but it never names them 'spam'. You interpret the clusters yourself.

Same raw data; the presence of labels changes the task, the algorithm, and how you judge results.

The rest of the map

Three more paradigms sit between and beyond these two:

Self-supervised learning is the honest workhorse behind modern foundation models: it is technically supervised, but the labels come free from the data (predict the masked or next word), so no human annotation is needed. Semi-supervised blends a small labeled set with a large unlabeled one. Reinforcement learning is a different setting entirely, an agent learns from reward signals, not a fixed dataset.

Further reading

• Google Machine Learning Crash Course, approachable, hands-on introduction to the core ideas.
• scikit-learn: Choosing the right estimator, a decision map from problem type to algorithm.
• scikit-learn: Supervised learning guide and Unsupervised learning guide, reference docs with runnable examples.