Inception

Inception is a family of convolutional neural network architectures for computer vision, whose first version, GoogLeNet, was a 22-layer deep network that won the great ImageNet visual recognition challenge of 2014. It was presented by a Google team led by Christian Szegedy, in a paper with a telling title: “Going Deeper with Convolutions.”

The innovation: the Inception module

Its central idea solved a recurring dilemma in network design: what filter size to use in each layer? The Inception module answers “all of them at once.” Instead of choosing one, it applies in parallel convolutions of several sizes (1×1, 3×3 and 5×5) and a pooling operation, and concatenates their outputs. Thus, a single layer captures patterns at different scales simultaneously.

The efficiency trick

Applying so many convolutions at once would be very expensive, so Inception introduced an ingenious saving: using 1×1 convolutions as “bottlenecks” that reduce the number of channels before the costly 3×3 and 5×5 convolutions. With that trimming, the network could be deep and powerful but surprisingly light in parameters, far fewer than other architectures of its time.

The name

The name has its charm. “GoogLeNet” is a tribute to LeNet, a pioneering network; and “Inception” alludes to the internet meme “we need to go deeper,” born from the film Inception—a nod to the network's depth that the authors themselves acknowledged in the paper.

Its evolution

The first version was followed by successive improvements. Inception v2 and v3 incorporated batch normalization and the factorization of convolutions (decomposing a 5×5 into two cheaper 3×3 ones). And Inception v4 and Inception-ResNet combined the Inception module with the residual connections popularized by ResNet, which speed up training.