Semi-Supervised Learning

Semi-supervised learning trains a model by combining a small amount of labeled data with a large amount of unlabeled data. It sits between supervised learning, which needs everything labeled, and unsupervised learning, which uses no labels; its aim is to exploit the structure of the unlabeled data to improve the classifier that would be obtained using only the few available labels.

When it works

It is not magic: it only helps if some assumption linking the data distribution to the labels holds. The main ones are smoothness (two points close together in a dense region tend to share a label), the cluster assumption (points in the same group tend to be of the same class, so the decision boundary passes through low-density areas) and the manifold assumption (the data concentrate on lower-dimensional structures). If those assumptions fail, semi-supervised learning may not improve—or may even worsen—over supervised learning.

Methods

There are three families. Pseudo-labeling, or self-training, uses the model's own most confident predictions as if they were true labels (Lee, 2013). Consistency regularization requires the prediction not to change under small perturbations of the same input; examples are Mean Teacher (2017) and FixMatch (2020), which combines consistency and pseudo-labeling with a confidence threshold. And graph-based methods propagate labels across a similarity graph between examples.

Not to be confused with self-supervised learning, and its risks

It should be distinguished from self-supervised learning, with which it is sometimes confused: the latter uses no human labels at all, inventing its own supervision from raw data, and it is what underpins the pre-training of foundation models. Semi-supervised learning, by contrast, does need a core of real labels. Its main risk is confirmation bias: if pseudo-labeling incorporates its own errors as if they were true, it tends to amplify them. That is why its advantage must be demonstrated empirically, not assumed.