Mahalanobis Distance

The Mahalanobis distance is a measure of distance, not an algorithm. Introduced by the Indian statistician Prasanta Chandra Mahalanobis in 1936, in his paper «On the Generalised Distance in Statistics», it quantifies how far a point lies from a distribution of data —or two points from each other relative to that distribution— while accounting for the covariance between variables. Unlike the Euclidean distance, which treats every direction equally, the Mahalanobis distance weights each direction according to how the data spread and correlate.

That weighting makes it invariant to scale and to correlation between variables: two quantities measured in wildly different units, or strongly correlated, no longer distort the calculation. It is therefore a statistical tool that other methods rely on, not a procedure that groups or classifies on its own.

What it measures and its formula

For a point x, a mean μ and a covariance matrix Σ, the distance is written in plain text as: d(x) = square root of (x − μ)ᵀ Σ⁻¹ (x − μ). The term Σ⁻¹, the inverse of the covariance matrix, is the crux: it rescales and «straightens» the space to cancel out correlations. When the variables are uncorrelated and normalized —that is, when Σ is the identity matrix— the formula collapses exactly into the Euclidean distance. The Euclidean distance is thus a special case of the Mahalanobis distance.

Why covariance matters

Ignoring covariance leads to faulty judgments. A point may look close to the center of a data cloud when measured in a straight line and still be an outlier if that cloud stretches in a particular direction. The Mahalanobis distance captures this geometry: it measures distance in «standard deviations» along the true axes of variation. Geometrically it amounts to applying a whitening transformation —a mapping through Σ⁻¹/² that decorrelates and normalizes the data— and then computing an ordinary Euclidean distance in the transformed space.

What it is used for and its link to PCA

Its applications are many: outlier and anomaly detection, quadratic discriminant analysis (QDA) and its use as the metric inside algorithms such as k-nearest neighbors (k-NN) or certain clustering methods. The nuance is worth stressing: the Mahalanobis distance can be the metric a clustering algorithm uses, but it never clusters anything by itself. Nor is PCA (Principal Component Analysis) a clustering algorithm: it is a dimensionality reduction technique. The two are related, but not through grouping: the whitening that underlies the Mahalanobis distance draws on the same decomposition of the covariance matrix on which PCA is built, hence the term «PCA whitening». Reducing dimensions and measuring distances are complementary tasks, not the same one.