Generative Models

A generative model is a machine-learning system that learns or approximates the probability distribution of a dataset and can use it to generate plausible new samples such as images, text, audio or video.

In classification, a discriminative model represents the conditional probability p(y|x)—‘which label belongs to this input’—whereas a generative one represents p(x) or the joint distribution p(x,y), making it possible to sample examples as well as classify them. Andrew Ng and Michael Jordan formalized the distinction in 2001 between generative classifiers, which model p(x,y), and discriminative classifiers, which model p(y|x); the ability to create samples comes from modeling a distribution, not from the label alone.

A generative model is a machine-learning model capable of producing new content, such as text, images, audio or video, from patterns learned during training.

The families: autoregressive, VAE, GAN and diffusion

Four broad families dominate the field. Autoregressive models generate a sample one element at a time, predicting the next from those already produced; this is the principle behind the GPT text models, which write word by word. Variational autoencoders (VAEs), proposed by Diederik Kingma and Max Welling in 2013, compress data into a probabilistic latent space that is then sampled to reconstruct new examples. Generative adversarial networks (GANs), introduced by Ian Goodfellow and colleagues in 2014, pit two networks against each other —a generator that fabricates samples and a discriminator that tries to catch them— in a game that sharpens the generator. And diffusion models, formalized by Jonathan Ho, Ajay Jain and Pieter Abbeel in 2020, add noise to data step by step and learn to reverse the process; today they underpin many image-generation systems. Each family has its own entry in this glossary. Documentation: Google technical glossary; original VAE paper; original GAN paper; original diffusion paper.

Why now: generative AI

Generative models are the engine of generative AI, one of the decade's most visible technology families. Mature architectures, large-scale compute and vast datasets have together moved systems that can write, draw, code or compose out of the lab and into everyday use. What were once academic demonstrations are now tools present in widely used products, and it is that ubiquity that has pushed generative AI to the center of the economic, cultural and regulatory conversation. Documentation: original foundation-model report; Google technical glossary.

Honest limits

Fluency should not be mistaken for reliability. Evaluating a generative model is inherently hard: there is no single correct answer to measure against, and automatic metrics capture real quality poorly. These systems also produce hallucinations —false statements delivered with confidence— and visual or audio artifacts that give the synthetic away. And deeper questions remain open about the data they are trained on and the authorship of what they generate, contested on technical, legal and ethical grounds. These are unresolved debates: we flag them here rather than settle them. Primary source. Primary source.