Artificial Neural Networks: Architecture and Applications
Artificial neural networks (ANNs) form the core of what is commonly known as "Deep Learning," a subdiscipline of artificial intelligence (AI) that draws insp...
Artificial neural networks are the engine of almost everything now called artificial intelligence. It is worth knowing how they are built, but above all worth seeing what they have achieved with figures, because that is the only way to tell a checkable accomplishment from a promise.
The architecture, in essence
A network is organized in layers of units — the «neurons» — connected by links carrying a weight, a number deciding how much each signal influences the next. There is an input layer, one or more hidden layers, and an output layer. Training means adjusting those weights so the output approaches what is expected.
That adjustment is done by gradient descent, and the algorithms governing it have improved considerably: stochastic descent with momentum, or Adam, find good values sooner and with less manual tuning.
Activation functions, and why they changed
Each unit applies an activation function to what it receives. For years the sigmoid and hyperbolic tangent were used, until a problem appeared in very deep networks: the vanishing gradient. Propagating the correction backwards layer by layer multiplies the signal many times by small numbers until it is practically zero, so the earliest layers stop learning.
The ReLU family — and its Leaky ReLU, parametric ReLU and ELU variants — solved much of this with a simple idea: let positive values through without compressing them.
The architectures and what each is for
- Convolutional (CNN): excel at visual tasks because they preserve the spatial hierarchy of the data — detecting edges first, then shapes, then objects.
- Recurrent (RNN): designed for sequences, they keep an internal state acting as memory of what has been seen.
- Attention and transformers: displaced recurrent networks in language by letting the model weigh all parts of the input at once, without traversing it in order.
Research keeps opening branches: capsule networks, seeking to represent hierarchical relationships in images better, or graph networks, extending all this to data that are neither tables nor sequences but relations between entities.
What they have achieved, with their authors' figures
Games: AlphaGo
AlphaGo, from DeepMind, combined convolutional networks to read the board with tree search. Its paper reports a 99.8% win rate against other Go programs and a 5-0 victory over the European champion — precision matters here: the 2016 Nature paper documents that match, and the encounter with world-class Lee Sedol came afterwards.
Language: BERT and GPT-3
BERT (Google, 2018) set the state of the art on eleven tasks, pushing the GLUE score to 80.5%. GPT-3 (OpenAI, 2020) reached 175 billion parameters, ten times more than any previous non-sparse model, and was applied without fine-tuning: the task was given in the input text itself.
Biology: AlphaFold
AlphaFold predicts a protein's three-dimensional structure from its amino acid sequence, one of biology's classic open problems. The database that made it useful worldwide is a collaboration between DeepMind and EMBL-EBI, the European Bioinformatics Institute, announced on 22 July 2021. It now holds over two hundred million predicted structures, openly accessible.
Autonomous driving: the example that does publish its numbers
Here the difference between companies in the sector is instructive. Waymo publishes its safety data with the denominator: through March 2026 it reports 220.6 million rider-only miles driven without a human driver, and compares its incidents with those of human drivers in the same areas — 94% fewer crashes with serious injury or worse, 82% fewer injury crashes and 93% fewer pedestrian injury crashes. And it points to three papers published in Traffic Injury Prevention supporting the methodology.
Those figures can be argued with — that is what the published denominator and methodology are for. What cannot be argued with is a figure nobody publishes. When a company gives the number, the period, the comparison area and the peer-reviewed method, it is making a checkable claim; when it gives only the result, it is advertising.
Who publishes the denominator and who does not
The previous section leaves a test useful across the whole sector, worth stating separately because it is the most useful thing in this article.
Waymo publishes the miles, the period, the comparison area and the peer-reviewed method. With that, anyone can argue with its figures: you can object that the miles accumulate in mild-weather cities, that the comparison with human drivers covers different journeys, or that the chosen period favours the system. All those objections are possible precisely because the data are published.
What cannot be argued with is a figure nobody publishes. And that is the asymmetry worth keeping in mind: when a company gives an improvement percentage without a denominator, it is not making a weak claim — it is making a claim that admits no checking, which is a different category.
The practical consequence is counterintuitive: the company publishing criticizable data is usually more reliable than the one publishing only conclusions. Exposing yourself to criticism costs something, and those who do it can generally afford to.
What a neural network cannot do by itself
One limit remains that none of the figures above resolves. All these systems learn from what already happened: games played, texts written, protein sequences measured, kilometres driven. Their competence rests on the future resembling the past they saw.
When that assumption fails — a traffic situation absent from the data, a document type nobody annotated, a protein with no known relatives — the system does not warn that it is off its territory. It keeps answering with the same confidence, because nothing in its design distinguishes «I have seen this a thousand times» from «I have never seen this».
That is the underlying reason these systems are deployed with human supervision wherever an error is costly. It is not distrust of the technology: it is that the architecture itself includes no mechanism for saying «I don't know».
What remains open
Two fronts. First, efficiency: networks are needed that require less compute and less labelled data, and semi-supervised learning and reinforcement learning push that way. Second, interpretability: we still cannot properly explain why a deep network decides what it decides, and that constrains its use anywhere a decision must be answered for.
The capability: how to read a neural network's achievement
1. Look for the denominator. «94% fewer crashes» only means something with the miles driven, the period and the comparison population beside it. Waymo provides them; they should always be demanded.
2. Check who it was compared against. AlphaGo beat the European champion in the 2016 paper, not the world champion. The difference takes nothing away: it places the result at its exact moment.
3. Verify who signs what. Proper names get mixed up easily — an institution quoted in a press release is not necessarily the one that did the work. Checking costs a minute and prevents repeating an error for years.
The deep end
- AlphaGo (2016), in the PDF DeepMind hosts itself. The Nature version is behind a paywall.
- BERT (2018) and GPT-3 (2020), both open on arXiv.
- AlphaFold database announcement (EMBL-EBI, July 2021).
- Waymo's safety data, with its cumulative figures and the studies behind them.
This article was produced with artificial intelligence under human editorial oversight.
