Generative Models

A generative model is a machine learning system that learns the probability distribution of a dataset in order to produce new, plausible samples from it: images, text, audio or video that resemble the training examples without copying any of them. Where a discriminative model only learns the boundary that separates one class from another —it models the conditional probability p(y|x), «which label fits this input»— a generative model captures the distribution of the data itself, p(x) or the joint p(x,y), which is why it can sample fresh examples rather than merely sort them.

This distinction, formalized by Andrew Ng and Michael Jordan in 2001, is the crux: learning to describe what the data look like, not just to tell classes apart, is what turns these models into machines for creation.

The families: autoregressive, VAE, GAN and diffusion

Four broad families dominate the field. Autoregressive models generate a sample one element at a time, predicting the next from those already produced; this is the principle behind the GPT text models, which write word by word. Variational autoencoders (VAEs), proposed by Diederik Kingma and Max Welling in 2013, compress data into a probabilistic latent space that is then sampled to reconstruct new examples. Generative adversarial networks (GANs), introduced by Ian Goodfellow and colleagues in 2014, pit two networks against each other —a generator that fabricates samples and a discriminator that tries to catch them— in a game that sharpens the generator. And diffusion models, formalized by Jonathan Ho, Ajay Jain and Pieter Abbeel in 2020, add noise to data step by step and learn to reverse the process; today they underpin the highest-quality image generation. Each family has its own entry in this glossary.

Why now: generative AI

Generative models are the engine of generative AI, the defining technological phenomenon of the decade. Mature architectures, large-scale compute and vast datasets have together moved systems that can write, draw, code or compose out of the lab and into everyday use. What were once academic demonstrations are now tools that millions of people reach for daily, and it is that ubiquity that has pushed generative AI to the center of the economic, cultural and regulatory conversation.

Honest limits

Fluency should not be mistaken for reliability. Evaluating a generative model is inherently hard: there is no single correct answer to measure against, and automatic metrics capture real quality poorly. These systems also produce hallucinations —false statements delivered with confidence— and visual or audio artifacts that give the synthetic away. And deeper questions remain open about the data they are trained on and the authorship of what they generate, contested on technical, legal and ethical grounds. These are unresolved debates: we flag them here rather than settle them.