Multi-Label Learning

Multi-label learning is the branch of machine learning that deals with problems where each example can carry several labels at once. The contrast with multi-class classification is the heart of the matter: a multi-class model picks exactly one category out of several mutually exclusive options — a photo shows a cat or a dog — whereas in multi-label learning the categories coexist: one news story can cover economics, technology and regulation, and one image can contain a beach, people and a sunset.

The standard representation is a binary label vector: each position is 1 if the label applies to the example and 0 otherwise. This is the formalisation adopted by the field's reference survey, by Min-Ling Zhang and Zhi-Hua Zhou in IEEE Transactions on Knowledge and Data Engineering (2014).

How it is tackled

The most direct strategy is Binary Relevance: train one independent binary classifier per label. It is sometimes written off as obsolete, yet it remains the field's baseline and is competitive and perfectly adequate for many problems, as the review by Zhang, Li, Liu and Geng in Frontiers of Computer Science (2018) documents. Its real limitation is that it does not model label dependencies: if beach makes sunset more likely, it cannot exploit that.

To capture those dependencies, the Classifier Chains of Read, Pfahringer, Holmes and Frank (Machine Learning, 2011) link binary classifiers so that each one receives the predictions of those before it; the same paper showed that binary-relevance-based methods scale to large datasets with strong performance. Label Powerset, covered in Tsoumakas and Katakis's overview (2007), treats every observed label combination as one class of a multi-class problem: it captures dependencies, but the number of combinations grows quickly. And adapted methods change the algorithm rather than the problem: decision trees, nearest neighbours or neural networks with a multi-label output. The dominant practice today is that last route: a network — often a transformer — with one sigmoid output per label.

How it is measured

No single metric tells the whole story. Hamming loss measures the fraction of individual labels predicted wrongly, averaged over examples and labels: it forgives partial mistakes. Subset accuracy, or exact match, only scores when the predicted label set matches the true one exactly; it is the strictest of all. F1 is averaged in two ways: micro, which pools hits and misses across all labels before computing and so weights frequent labels more, and macro, which averages each label's F1 equally and gives rare labels visibility.

One piece is often forgotten: decision thresholds. The model outputs one probability per label, and deciding which labels fire requires a threshold — 0.5 by default, tunable globally or per label depending on the metric being optimised. The threshold is as much part of the system as the network's weights.

Where it is used today

Its classic grounds are text and image tagging and functional genomics, where a single protein can perform several biochemical functions. The current frontier is extreme multi-label classification (XMLC), with label spaces in the hundreds of thousands or millions, as in Wikipedia tagging or product recommendation; systems such as X-Transformer, by Chang and colleagues (KDD, 2020), showed how to fine-tune transformers for spaces of half a million labels. Open fronts remain: long-tail labels with very few examples, threshold calibration at that scale, and modelling dependencies when labels number in the millions.