Zero-Shot Learning

Zero-shot learning (ZSL) is a model's ability to solve a task or recognize a class for which it saw no examples during training. Unlike classic supervised learning, which needs many labeled examples of every category, ZSL asks the system to generalize to what it never observed by leaning on some form of auxiliary knowledge.

The question it answers is blunt: how can a machine identify something that was absent from its data? The trick is to bridge the seen and the unseen through semantic information.

The classic approach: semantic attributes

In computer vision, ZSL was built on human-defined auxiliary information. Instead of learning each class in isolation, the model learns reusable attributes —has stripes, is brown, lives in water, eats fish— and describes each category as a combination of them. It can then recognize a zebra it has never seen if it knows the animal is, in essence, «a horse with stripes».

Mark Palatucci, Dean Pomerleau, Geoffrey Hinton and Tom Mitchell formalised this bridge through semantic output codes that connect seen and unseen classes.

Zero-shot in large models: GPT-3 and CLIP

The term returned forcefully in the era of large models, with a broader meaning. In 2020, Tom Brown's team at OpenAI presented GPT-3 in «Language Models are Few-Shot Learners»: describing a task in plain language, with no examples, was enough for the model to carry it out. This is zero-shot prompting.

In 2021, Alec Radford and colleagues took the idea multimodal with CLIP («Learning Transferable Visual Models From Natural Language Supervision»). CLIP classifies an image by comparing it against arbitrary text descriptions, so it can label new categories without retraining —simply by changing the candidate phrases. Documentation: original GPT-3 paper; original CLIP paper.

Against one-shot and few-shot, and its limits

Degrees matter. One-shot provides a single example of the class or task; few-shot, a handful; zero-shot, none. More examples usually mean better results: zero-shot almost always trails fine-tuning with examples.

An honest debate stays open: how much of zero-shot success is genuine generalization to the unseen, and how much reflects that supposedly «new» classes or tasks appeared, in part, within the vast pretraining corpora. Measuring that boundary rigorously remains an active research problem. Documentation: foundational zero-shot paper; GPT-3 evaluation.