Classification

Classification is a supervised-learning task in which a model learns, from labeled examples, to assign each input to one of a finite set of categories or classes. It differs from regression, whose output is a continuous number rather than a category.

Types of classification

Three types are worth distinguishing. Binary has two mutually exclusive classes, such as “spam” or “not spam.” Multiclass has more than two, but each example gets only one: a handwritten digit is a 7 and not at the same time a 3. And multilabel allows assigning several labels—or none—to the same example, because they are not exclusive: an image may contain both “dog” and “cat.”

Algorithms

There are no few canonical “pillars,” but a range of algorithms chosen according to the problem: logistic regression, k-nearest neighbors, decision trees and random forests, support vector machines, naive Bayes, gradient boosting and neural networks. A clarification is in order: generative adversarial networks (GANs) are not classifiers, but generative models; their relation to classification is indirect, as a tool for augmenting the training data.

How it is evaluated

The most intuitive metric is accuracy (the proportion of correct predictions), but it misleads with imbalanced classes: if 99% of cases belong to one class, always predicting that class is right 99% of the time without being useful. That is why precision, recall and their harmonic mean, the F1 score, are used, along with the confusion matrix and the area under the ROC curve, which give a truer picture when the classes are not balanced.