The Most Popular Supervised Learning Algorithms and Their Applications
The field of supervised learning, a subcategory of artificial intelligence (AI), has seen exponential growth, evidenced by the diversification of algorithms ...
Supervised learning — teaching a model from already labelled examples — underpins much of what artificial intelligence actually does today, far from the headlines. Here are its four main families, what each is for, and one case that teaches how to tell a demonstrated capability from a promise.
Decision trees and random forests
A decision tree is a structure of chained questions: at each node the dataset is split by a condition, and the leaves hold the answer. Its great virtue is that it can be read: anyone can follow the path that led to a particular prediction.
A single tree is fragile, because it overfits the data it learned from. Random forests solve that by building many trees on different samples and combining their predictions. They gain accuracy and robustness, and pay a price worth knowing: they lose the readability of the single tree. A forest of five hundred trees can no longer be read by following one path.
It is the most recurrent trade-off in the whole field, and it will come up again and again: accuracy in exchange for being able to explain.
Support vector machines
SVMs look for the boundary that best separates two classes, maximizing the margin between them. When that boundary is not a straight line, they use what is called the kernel trick: projecting the data into a higher-dimensional space where a clean separation does exist, without computing that projection explicitly. They work well with many variables and few examples, a common scenario in biomedicine.
The quantum kernel case: how to read a promise
This deserves a pause, because it is usually told badly and it teaches how to check any emerging technology.
There is a research line proposing to compute those kernels on a quantum computer. The reference work is «Supervised learning with quantum-enhanced feature spaces», by a team from IBM and MIT, published in Nature in 2019 — the journal version sits behind a paywall, but the team keeps the preprint open.
And now the experiment's real dimensions, which almost never accompany the citation: they used two qubits of a five-qubit processor, two variables, and sets of twenty points per label for training and twenty more for testing. The data, moreover, were synthetic, generated on purpose to be separable by their own method: in their words, «we generate artificial data that can be fully separated by our feature map».
That is an impeccable and honest proof of principle. It is not — and their paper does not claim it is — a technology that today solves classification problems beyond the reach of a classical SVM.
And what happens when someone measures it seriously? In 2024 a team published «Better than classical? The subtle art of benchmarking quantum machine learning models», a preprint comparing twelve common quantum models across 160 datasets. Their conclusion, verbatim: «overall, out-of-the-box classical machine learning models outperform the quantum classifiers». And a finding still more awkward for the sector's marketing: «removing entanglement from a quantum model often results in as good or better performance». That is, quantumness did not appear to be the decisive ingredient.
The most cited assessment of the matter, this one peer-reviewed, is by Maria Schuld and Nathan Killoran in PRX Quantum (2022): quantum computing «does not offer practical benchmarks on realistic scales, and theory is the main tool we have».
Honest conclusion: quantum kernels are an interesting theoretical promise without practical demonstration at a scale that matters. Whether they may become one is an open question; that they already are, is not.
Neural networks and deep learning
Neural networks stack layers of units that transform the input until they produce an output. With many layers — deep learning — they extract increasingly abstract patterns without anyone designing them by hand. It is the family that dominates when the data are images, audio or text, and the hardest of the four to explain.
Boosting and XGBoost
Boosting builds weak models in sequence, each correcting its predecessor's errors, until they form a strong one. XGBoost is its most widespread implementation and remains the reasonable first choice when data come in tabular form — rows and columns, as in finance or insurance — a terrain where neural networks have not displaced these methods.
That fact is useful on its own: deep learning did not win everywhere. On tabular data, tree ensembles still compete head to head.
How to choose among the four
Lined up like this, the choice stops being a matter of fashion and comes down to the shape of the data and what has to be answered afterwards.
If the data arrive as a table — rows and columns, like a customer base or a claims history — the first reasonable option remains tree ensembles: boosting or random forests. If they are images, audio or text, deep learning wins, extracting for itself the features nobody could enumerate. If there are many variables and few examples — common in biomedicine — support vector machines remain competitive. And if what is needed is being able to explain each decision, a single tree or a linear regression is worth more than a better model nobody can read.
That last condition is no concession: in a bank, a hospital or a public administration, a model that cannot be justified cannot be used, however good it is.
And how to look inside when there is no alternative
When the problem demands a barely readable model, techniques exist to reconstruct why it decided as it did. The two reference ones are LIME, which approximates the model with a simple one around a specific prediction, and SHAP, which distributes responsibility across variables using cooperative game theory's Shapley values.
It pays to know what they are and are not. They are approximations of the model's behaviour, not a record of what it did. They serve to detect that a system leans on a variable it should not — a postcode where it should be reading income — and they do not serve as a guarantee that the decision was right. They are a tool for suspicion, not for absolution.
The capability: telling a demonstrated capability from a promise
The quantum kernel case leaves a method that works for any emerging technology:
1. At what scale was it demonstrated? Two qubits and forty points is not a real problem. The experiment's scale is usually in the paper and almost never in the headline.
2. Were the data real, or manufactured for the demonstration? A method tested on data generated so that this same method separates them demonstrates internal consistency, not usefulness.
3. Is there a broad comparison against the conventional approach? One isolated favourable result says little against a benchmark with dozens of datasets. And if nobody has run that comparison, the absence is information too.
The deep end
- Havlíček and others (2018/2019), the quantum kernel experiment. The section describing the data is short and worth reading in full: it is an example of a paper that states its own limits clearly.
- Bowles, Ahmed and Schuld (2024), the benchmark across 160 datasets. It is a preprint, not a peer-reviewed paper, and should be cited as such.
- Schuld and Killoran (2022), published in PRX Quantum, on whether quantum advantage is even the right goal.
This article was produced with artificial intelligence under human editorial oversight.
