Unsupervised Learning

Unsupervised learning is the branch of machine learning that looks for structure in data without labels. The model is never told the right answer; it must discover regularities, groups and patterns on its own. This sets it apart from supervised learning, which learns from already-labelled examples (this image is a cat), and from reinforcement learning, which learns from rewards earned by acting in an environment. Where supervised learning imitates and reinforcement learning experiments, unsupervised learning explores.

In the language of the scikit-learn documentation and the classic textbooks (Bishop; Goodfellow, Bengio and Courville), its raw material is the unannotated example and its goal is to model how the data is distributed or organised.

The tasks

Four families cover almost everything. Clustering gathers similar examples into groups: k-means, DBSCAN, hierarchical methods. Dimensionality reduction compresses many variables into a few while keeping the essentials, through techniques such as PCA or t-SNE, useful both for visualisation and for feeding downstream models. Density estimation models the probability of the data and underpins anomaly detection: the rare is simply the improbable. And representation learning seeks compact, rich encodings —autoencoders— that capture latent meaning. IA360 keeps separate entries for clustering and dimensionality reduction; here it is enough to see them as pieces of one map.

Self-supervision and foundation models

The defining shift of the past decade has a name: self-supervised learning. Yann LeCun popularised it with his «cake analogy» —the bulk of the cake is unsupervised learning, the icing is supervised learning and the cherry is reinforcement learning— and in 2019 he swapped «unsupervised» for «self-supervised», which he described as a variant in which the data itself provides the supervision. The trick is to manufacture labels from the data. Hiding a word and predicting it (BERT's masked language modelling) or predicting the next token (the GPT family) turns unannotated text into millions of exercises that come with their own answer. That pretraining is now the engine of foundation models: the Stanford report that coined the term (Bommasani et al., 2021) defines them as models trained with self-supervision at scale on broad data and then adapted to a wide range of downstream tasks.

Uses and limits

It serves to segment customers, flag fraud and anomalies, compress information and, above all, pretrain the large language and vision models. Its honest limit: without labels, evaluation is hard. There is no correct answer to measure against, so the quality of a clustering is judged by internal metrics or by downstream usefulness, and the worth of a representation is measured by what it later enables. How to compare two unsupervised solutions objectively, when neither carries a ground truth, remains an open and unsolved problem.