Spiking Neural Networks
Spiking neural networks (SNN) are biology-inspired models in which neurons communicate via discrete impulses in time. We explain how they work, the challenge of training them, and their efficiency on neuromorphic hardware, with the caveat that they do not replace conventional deep networks.
A spiking neural network (SNN) is a biology-inspired neural network model in which neurons communicate through discrete impulses—spikes—emitted at specific instants in time, rather than exchanging continuous values at every step. Its essential feature is that it incorporates time: information lies not only in how much a neuron fires, but in when.
How they work
The most common neuron model is the leaky integrate-and-fire. Each neuron accumulates the incoming current in a variable, the membrane potential, which also decays over time if it receives no stimulus; when that potential exceeds a threshold, the neuron emits an impulse and resets. Computation is event-driven and asynchronous: there is activity only when an impulse occurs, and that sparsity is the basis of its efficiency.
The challenge of training them
The central obstacle is that the spike is not differentiable—it is a step—which prevents applying backpropagation directly. There are three routes: surrogate gradients, which replace the step's derivative with a smooth function during the backward pass and are today the dominant approach; the conversion of an already-trained conventional network into a spiking one; and biology-inspired rules such as STDP, in which a synapse is strengthened or weakened according to the temporal order of the spikes.
Where they shine, and a caveat
Their great asset is energy efficiency when run on neuromorphic hardware: Intel's Loihi uses sparse computation driven by events so as to minimize both system activity and the movement of data; IBM's TrueNorth brings together one million digital neurons in a chip whose measured power draw is 65 mW; and SpiNNaker runs with an asynchronous model in which events drive processing. They stand out in event-based vision, robotics and low-power edge sensors. An honest caveat is in order, though: spiking networks are not “the latest generation” replacing Transformers, but a distinct line, oriented toward efficiency and low latency, that still lags conventional deep networks on many accuracy tasks.
Pieces using this term
This article was produced with artificial intelligence under human editorial oversight.
