Supervised Learning

Supervised learning is the machine-learning paradigm in which a model learns from labeled data—input and desired-output pairs—to map new inputs to their correct output. It is called “supervised” because the labels act as a teacher that corrects: the algorithm compares its prediction with the known answer and adjusts its behavior.

Classification and regression

Depending on the type of output, two tasks are distinguished. In classification, the output is a discrete category: whether an email is spam or not, whether a tumor is benign or malignant, which digit is in an image. In regression, the output is a continuous number: the price of a house or tomorrow's temperature.

How it works

The model is fit by minimizing a loss function that measures the error between its predictions and the true labels on the training data. But the goal is not to be right on the training set, but to generalize to new data; that is why the data are split into a training set, a validation set—to tune and monitor—and a test set, reserved for a final, unbiased estimate. If the training error falls while the validation error rises, the model is overfitting: memorizing instead of learning.

Against other approaches, and its limits

Unlike unsupervised learning, which works without labels, or reinforcement learning, guided by rewards, supervised learning depends on having labels, and labeling is costly and slow; hence the emergence of semi-supervised and self-supervised learning. Its representative algorithms range from logistic regression, k-nearest neighbors, support vector machines or trees and forests to neural networks. And its risks are overfitting and the biases the model inherits from biased labels.