Models

In machine learning, a model is the representation that a learning algorithm produces from data: a set of parameters —the adjusted weights— together with a structure, used to make predictions or decisions about new data. Google's machine learning glossary puts it plainly: a model is the set of parameters and structure a system needs to make predictions.

This yields the single most important distinction, and the one most often blurred: the algorithm is not the model. The algorithm is the training procedure —the method that sweeps through the data and tunes the parameters—; the model is its output, the weights once fixed. Logistic regression, gradient descent or a decision tree are algorithms; what remains after you train them on a particular dataset is a model. The same algorithm, given different data, yields different models.

Parametric and non-parametric, discriminative and generative

Models are classified by how they structure what they learn. A parametric model compresses the data into a fixed number of parameters: a linear regression stores only its coefficients, however many examples it sees. A non-parametric model —k-nearest neighbours, many tree-based methods— lets complexity grow with the data; in statistical terms, its parameter space is infinite-dimensional.

Another classic split separates the discriminative from the generative. A discriminative model learns the boundary between classes directly, that is, the conditional probability P(y|x): logistic regression is the textbook case. A generative model learns how the data is produced —the joint distribution P(x,y), or P(x)— and derives its decisions from there; the naive Bayes classifier is the historical example, and today's large text and image generators are its heirs at a different scale.

Every model is a simplification

A model is never reality, only an idealised version of it. The statistician George Box coined the line in 1976: "all models are wrong, but some are useful." Usefulness turns on the fit. An underfit model is too simple and misses the pattern even in the training data; an overfit one memorises the noise in that data and fails on new examples. The balance between the two extremes is generalisation: the real ability to be right about data the model never saw —the only thing that matters once it leaves the lab.

From model to deployment

Training a model is only half the work. Once its parameters are fixed, it is evaluated on held-out data, deployed for inference, and monitored, since its behaviour can degrade when reality drifts away from the data it learned on. A special case is the foundation model —named by Stanford University in 2021—: a model trained at scale on broad data and reusable across many tasks. It shares this very definition, but its size and impact earn it an entry of its own.