Multilayer Perceptron
The multilayer perceptron is a feedforward network with hidden layers and nonlinear activations, trained by backpropagation. We correct a widespread error—Hinton did not create it; its roots are in Rosenblatt (1958)—clarify that “multilayer” is not “deep,” and show it lives on inside Transformers.
The multilayer perceptron (MLP) is a feedforward—acyclic—fully connected artificial neural network, organized in layers: an input layer, one or more hidden layers and an output layer, with nonlinear activation functions and trained by backpropagation. Its minimum requirement is to have at least one hidden layer.
A historical correction
A widespread error is worth undoing: the multilayer perceptron was not created by Geoffrey Hinton. Its roots lie in the perceptron of Frank Rosenblatt, who formulated it in 1958, and the very idea of stacking several layers predates Hinton. What Hinton did, together with David Rumelhart and Ronald Williams, was to popularize in 1986 the backpropagation algorithm, which made it possible to train networks with hidden layers effectively; that method also had antecedents in the work of Paul Werbos (1974) and in the reverse-mode differentiation of Seppo Linnainmaa (1970). In other words, Hinton did not invent the MLP: he helped make its training practical.
Multilayer is not the same as deep
Another clarification: “multilayer” does not imply “deep.” A single hidden layer is enough for a legitimate MLP, and such a network is shallow. “Deep” refers to many stacked hidden layers: every deep MLP is multilayer, but not every MLP is deep.
How it works
Each neuron computes a weighted sum of its inputs plus a bias and applies a nonlinear activation function. That nonlinearity is essential: without it, stacking layers would add nothing, because the composition of linear transformations is still linear and the network would amount to a single linear model. The universal approximation theorem (Cybenko, 1989; Hornik, 1991) shows that an MLP with a single hidden layer and enough neurons can already approximate any continuous function over a bounded domain; depth helps efficiency, but is not a requirement for that expressiveness. Training by backpropagation propagates the error from the output backward, via the chain rule, to adjust the weights.
Its place today
The MLP is the canonical building block of the fully connected network and remains very much alive: inside every layer of a Transformer there is a block that is, literally, an MLP applied at each position—two linear transformations with a nonlinear activation in between. So this idea, with roots in the 1950s, also underpins today's large language models.
This article was produced with artificial intelligence under human editorial oversight.
