Siamese Networks

A Siamese network is an architecture made of two —or more— identical subnetworks that share exactly the same weights and each process a different input. Rather than sorting an input into categories, its aim is to compare: it learns to measure how alike or unlike two inputs are, mapping them into an embedding space where the distance between the vectors reflects their similarity.

Sharing the weights is the crux: it guarantees that both inputs are transformed by the same function, so that two matching items yield nearby vectors and two different ones yield distant vectors. The network does not learn «what» each input is, but «how alike» two of them are.

How it learns: embeddings and comparison losses

A Siamese network is not trained with the usual classification loss but with functions that operate on pairs or triplets of examples. The contrastive loss, formalized by Raia Hadsell, Sumit Chopra and Yann LeCun in 2006, pulls together in embedding space the pairs that should match and pushes apart those that should differ, up to a margin. The triplet loss works with three inputs at once: an anchor, a positive example (same class as the anchor) and a negative one (a different class); training drives the anchor closer to the positive than to the negative by a set margin. The result is a space where distance alone already encodes identity.

Uses: face and signature verification, and one-shot

The idea was born precisely to compare. Jane Bromley, Isabelle Guyon, Yann LeCun and colleagues introduced Siamese networks in the early 1990s for handwritten signature verification, deciding whether two signatures belonged to the same person. Its best-known application is face verification: Google's FaceNet (Florian Schroff and colleagues, 2015) used the triplet loss to learn face embeddings that confirm whether two photos show the same person. And in one-shot learning, Gregory Koch, Richard Zemel and Ruslan Salakhutdinov (2015) showed that a Siamese network can recognize a new class from a single example by comparing it against the reference. These applications have their own entries in this glossary.

Why it works: metric learning with many classes and few examples

Siamese networks are a form of metric learning: instead of learning boundaries between classes, they learn a distance measure. That difference makes them ideal when there are a great many classes and very few examples of each —millions of distinct faces or signatures, with barely one or two samples per identity—, a setting where a conventional classifier is unworkable because it would have to be retrained every time a new identity appears. A Siamese network, by contrast, generalizes: once it has learned the notion of «similarity», it can compare faces or signatures it never saw during training. Open questions remain in how the training pairs or triplets are chosen —«hard negative» mining strongly affects quality— and in the way embeddings can inherit biases present in the data.