Online Learning

Online learning is the machine-learning regime in which data arrive sequentially and the model updates its prediction with each item or small batch it receives, rather than training once on a complete, fixed set, as in batch learning. It is especially useful when data form a continuous stream, when they arrive too fast to store, or when the environment changes over time.

It is worth stressing that this is a learning protocol, not a family of models: online learning can be done with linear models, a perceptron, trees or neural networks.

Its theoretical basis

The correct framework for understanding it is online convex optimization and the minimization of regret: the difference between the model's cumulative loss and that of the best fixed predictor that could have been chosen in hindsight. An algorithm is good if that regret grows more slowly than the number of rounds, so that, in the long run, the model performs as well as that best predictor. The foundational algorithms are online or stochastic gradient descent, the perceptron, and variants such as Follow-the-Regularized-Leader. It is a mistake to tie online learning to the universal approximation theorem: that theorem concerns which functions a network can represent, a separate matter, independent of the order in which data arrive and of how the model is updated.

Online, incremental and continual

Three close terms should be kept apart. Online learning emphasizes immediate, sequential updating with bounded memory. Incremental learning is nearly a synonym, emphasizing building the model progressively without retraining on the whole history. Continual, or lifelong, learning faces a sequence of different tasks or distributions and seeks to accumulate knowledge without forgetting what was learned.

Concept drift

The challenge specific to real streams is concept drift: the change in the data distribution over time. If the relationship between inputs and the target variable changes—so-called real drift—accuracy degrades and the model must adapt; it can do so passively, with a sliding window or a forgetting factor, or actively, detecting the change and reacting. In continual learning there also arises catastrophic forgetting: when learning new tasks, the network overwrites the weights important for earlier ones. The nuance here is subtle: under concept drift, forgetting the old is desirable because it is no longer valid; in continual learning, forgetting tasks that remain valid is precisely the failure to be avoided.