Models

In machine learning, a model is the representation that a learning algorithm produces from data: a set of parameters —the adjusted weights— together with a structure, used to make predictions or decisions about new data.

This yields the single most important distinction, and the one most often blurred: the algorithm is not the model. The algorithm is the training procedure —the method that sweeps through the data and tunes the parameters—; the model is its output, the weights once fixed. Logistic regression, gradient descent or a decision tree are algorithms; what remains after you train them on a particular dataset is a model. The same algorithm, given different data, yields different models. Primary source.

Parametric and non-parametric, discriminative and generative

Models are classified by how they structure what they learn. A parametric model compresses the data into a fixed number of parameters: a linear regression stores only its coefficients, however many examples it sees. A non-parametric model —k-nearest neighbours, many tree-based methods— lets complexity grow with the data; in statistical terms, its parameter space is infinite-dimensional. Primary source.

Another classic split separates the discriminative from the generative. A discriminative model learns the boundary between classes directly, that is, the conditional probability P(y|x): logistic regression is the textbook case. A generative model learns how the data is produced —the joint distribution P(x,y), or P(x)— and derives its decisions from there; the naive Bayes classifier is the historical example, and today's large text and image generators are its heirs at a different scale. Primary source.

Every model is a simplification

A model is never reality, only an idealised version of it. Usefulness turns on the fit. An underfit model is too simple and misses the pattern even in the training data; an overfit one memorises the noise in that data and fails on new examples. The balance between the two extremes is generalisation: the real ability to be right about data the model never saw —the only thing that matters once it leaves the lab. Primary source.

From model to deployment

Training a model is only half the work. Once its parameters are fixed, it is evaluated on held-out data, deployed for inference, and monitored, since its behaviour can degrade when reality drifts away from the data it learned on.