pLSA

The pLSA (Probabilistic Latent Semantic Analysis) is a statistical latent-variable model for topic modeling, introduced by Thomas Hofmann in 1999. Its aim is to uncover the hidden topics that organize a collection of documents and to explain, in probabilistic terms, why certain words tend to appear together in some texts and not in others.

Where classical latent semantic analysis factorizes a matrix through linear algebra, pLSA rests on a probabilistic foundation: it models the co-occurrence of words and documents through a latent topic variable, usually written as z.

How it works

pLSA relies on a bag-of-words representation: it ignores word order and counts only how often each term occurs. Under the model, every document is a mixture of topics, and every topic is a probability distribution over the vocabulary. The probability of seeing a word in a document is expressed as a sum over topics: a topic z is drawn according to its weight in the document, and the word is then generated from that topic. The parameters —which words define each topic, and in what proportion each topic appears in each document— are estimated by maximizing the likelihood of the corpus with the expectation-maximization (EM) algorithm, which alternates between estimating topic probabilities and readjusting the parameters.

Its limit against LDA: overfitting and no prior

pLSA's central weakness is that it is not a complete generative model at the document level. The topic proportions of each document are parameters estimated one by one, so their number grows linearly with the size of the corpus. This surplus of parameters causes overfitting and, just as importantly, leaves the model with no natural way to assign probability to a new document that was absent from training. Blei, Ng and Jordan solved both problems in 2003 with Latent Dirichlet Allocation (LDA): they placed a Dirichlet prior over the topic mixtures, so that these are no longer fixed parameters but random variables governed by a handful of hyperparameters. The parameter count no longer depends on the number of documents, and the model can generalize to unseen text.

Where it stands today

pLSA was the conceptual link that made modern topic modeling possible, yet in practice it has been superseded. LDA became the standard for nearly two decades, and it now coexists with neural, embedding-based methods such as BERTopic (Maarten Grootendorst, 2022), which groups documents by semantic similarity using transformer representations and describes each group with a class-based TF-IDF. Which approach performs best remains unsettled: the advantage of neural methods over LDA depends on the corpus, the language and the task, and is still debated.