One-Shot Learning

One-shot learning is the problem of teaching a system to recognize a new class from a single labeled example, or from very few. It stands in contrast to classic deep learning, which needs thousands or millions of images per category to reach good accuracy. The inspiration is human: a child needs to see an animal only once to recognize it afterward.

It helps to place it on a spectrum. In zero-shot there is no example of the class at all, and recognition relies on a description or on attributes; in one-shot there is exactly one; in few-shot, a handful. The pioneering work was the Bayesian approach of Fei-Fei Li, Rob Fergus, and Pietro Perona (2006), which learned object categories from one or a few images by reusing what had been learned from other categories as prior knowledge.

Why a single example changes everything

With just one example, standard methods break down: there is not enough data to estimate how much the class varies, and a model trained directly on that single point overfits and fails to generalize. The dominant answer is not to learn the class but to learn a metric: a function that measures how similar two examples are. If you learn a space where objects of the same class land close together and objects of different classes land far apart, classifying a new image reduces to comparing it against the one available example and keeping the closest match. Deep learning supplies the embedding; distance supplies the decision.

Compare, don't memorize: siamese, matching, and prototypical

The siamese networks of Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov (2015) were a landmark: two copies of the same network process two images and learn to decide whether they belong to the same class, evaluated on the Omniglot dataset. The Matching Networks of Oriol Vinyals and colleagues (2016) map a small support set and an unlabeled example directly to its class, with no retraining. The Prototypical Networks of Jake Snell, Kevin Swersky, and Richard Zemel (2017) condense each class into a «prototype» —the mean of its examples in the learned space— and classify by proximity. All of this connects to meta-learning, or «learning to learn»: MAML, by Chelsea Finn, Pieter Abbeel, and Sergey Levine (2017), seeks an initialization from which a model can adapt to a new task in very few steps and examples.

The «one-shot» of large language models

Since 2020, «one-shot» has carried a related but different meaning in LLMs. In «Language Models are Few-Shot Learners», Tom Brown and colleagues (GPT-3) call it one-shot when a single example is placed inside the prompt —in-context learning— as opposed to zero-shot (instruction only) or few-shot (several examples). The difference is crucial: nothing is trained here and no weights are updated; the model exploits an example it sees on the fly. So one should not confuse «training on one example» with «showing one example in the prompt»: two ideas that share a name and work in opposite ways. In practice, classic one-shot shines in facial recognition and verification, where there is often only one reference photo per person.