Topic Models

Topic models are unsupervised learning algorithms that automatically discover the latent “themes” running through a large collection of documents, by analyzing which words appear together and without the need for labeled data. They serve to bring order to text corpora that would be unmanageable by hand.

The core idea

Topic modeling rests on a twofold probabilistic structure: each topic is a probability distribution over the words of the vocabulary (the topic “economy” will give high weight to “market,” “price” or “inflation”), and each document is a mixture of topics in different proportions. A single text may be 70% about politics and 30% about technology.

The canonical model and its antecedents

The reference model is LDA (Latent Dirichlet Allocation), introduced by David Blei, Andrew Ng and Michael Jordan in 2003. Its authors described it as “a generative probabilistic model for collections of discrete data such as text corpora.” Two key antecedents came before, born in information retrieval: LSA/LSI (Deerwester and colleagues, 1990), based on linear algebra (singular value decomposition) and not probabilistic, and Thomas Hofmann's pLSA (1999), which already introduced latent topic variables.

How they are interpreted and used

A topic is interpreted by looking at its most probable words, and from there the analyst labels it. They are used to organize and explore large document archives, discover thematic trends and support information retrieval. It is worth recalling a classic caveat—raised by Chang and colleagues in 2009, in a paper titled “Reading Tea Leaves”—: automatic quality metrics do not always match what a person considers a coherent topic, so interpretation requires human judgment.

From LDA to transformers

LDA remains valid where explainability and low computational cost matter, but the landscape has changed. Recent approaches, such as BERTopic (2022), represent each document with embeddings generated by transformer-based language models, cluster them and derive the topics from there; they tend to handle short texts better and capture context and semantics, something beyond the reach of classic LDA's word counting.