Underfitting

Underfitting happens when a model is too simple to capture the underlying structure of the data. Because it starts from assumptions that are too rigid —fitting a straight line to points that clearly trace a curve, for instance— it fails to learn even the regularities present in the training set. The symptom is unmistakable: the model performs poorly on both the training data and the test data.

In machine-learning terms, underfitting is the visible face of high bias: the model arrives with strong assumptions about the shape of the relationship, and those assumptions keep it from getting close to the truth. As IBM's documentation puts it, a high-bias model is too simplistic to reflect the real patterns, so it stumbles on the training set itself.

Against overfitting: the bias-variance tradeoff

Underfitting is the opposite of overfitting. Where an underfit model ignores the signal, an overfit one memorizes the noise: it learns the training set beautifully but fails to generalize to new data. Both extremes are explained by the bias-variance tradeoff. In the classic Deep Learning textbook by Goodfellow, Bengio and Courville, model capacity frames the tension: too little capacity and bias dominates, producing underfitting; too much and variance explodes, producing overfitting. The Elements of Statistical Learning, by Hastie, Tibshirani and Friedman, formalizes that decomposition of the error. The goal is not to minimize either term but to find the sweet spot of capacity that generalizes well.

Common causes

Underfitting usually has concrete roots: a model that is not expressive enough for the task (a line for curved data, a tree that is too shallow); insufficient or poorly chosen features that fail to supply the information the model needs; excessive regularization, which penalizes complexity so heavily that the model turns anemic; or insufficient training, stopped before the model converges. Note the difference between overusing regularization, which underfits, and using it well, which fights overfitting.

How to diagnose and fix it

The diagnosis is direct and sets it clearly apart from overfitting: a high training error. If the model already fails on data it has seen, capacity is missing; in overfitting, by contrast, training error is low while test error is high. Learning curves —like those in scikit-learn's polynomial-regression example— help confirm the picture. The remedies follow the cause: adopt a more complex or flexible model, add or improve features through feature engineering, reduce regularization, or train for longer. Once underfitting is fixed, attention shifts to the opposite end so the model does not tip into overfitting.