Euclidean distance: when “close” means something

On January 12, 1999, Vladimir Pestov submitted a paper connecting similarity search with concentration of measure in high-dimensional spaces. The date matters because this problem did not begin with today’s AI models: a rule as familiar as Euclidean distance can lose discriminating power as a representation grows. Before accepting that two records are “close,” ask what each coordinate represents, which unit it uses and whether a straight line corresponds to the similarity that matters.

The formula measures a diagonal, not meaning

Given two vectors with the same number of coordinates, Euclidean distance subtracts each pair of values, squares every difference, adds the squares and takes the square root. For points (2, 10) and (5, 14), the differences are 3 and 4; the squared sum is 9 + 16 = 25, so the distance is 5. It generalizes the Pythagorean theorem: straight-line length inside the selected coordinate system.

The result has the properties of a metric. It is never negative; it is zero between a point and itself; swapping the points does not change it; and the direct route is no longer than a route through a third point. That final condition is the triangle inequality. These properties support reasoning about neighborhoods and pruning search regions, but they do not establish that the coordinates suit the problem.

Squared Euclidean distance is sometimes enough. Removing the square root does not change distance rankings because square root is increasing over nonnegative values. It can identify the same nearest neighbor with less work. Yet the squared quantity is not itself a metric: on a line, the squared distance from 0 to 2 is 4, while going through 1 adds 1 + 1. The two quantities should not be exchanged merely because their rankings match.

Scale decides which coordinate rules

Suppose a system compares customers using annual income in euros and tenure in years. An income difference of 10,000 euros contributes 100,000,000 to the squared sum; a difference of 10 years contributes 100. Without a transformation, income overwhelms the distance even if tenure is equally or more important to the decision. The formula has not failed. It has executed the weighting implicit in the units.

A common response is to standardize every feature by subtracting its mean and dividing by its standard deviation. The StandardScaler documentation says it computes those statistics from training samples and warns that a feature whose variance is orders of magnitude larger may dominate an objective. It also names a limit: the method is sensitive to outliers. Standardization does not make features equivalent by decree; it changes the question to differences measured in standard deviations.

The transformation must be learned from the training set and then applied without recomputing it on held-out evaluation cases. Otherwise, the mean and scale incorporate information from the future exam. Scaling is also a weighting choice: dividing one coordinate by two cuts its squared contribution to one quarter. An audit should preserve the parameters, the population used to estimate them and the reason for each transformation.

Sharing columns does not guarantee shared geometry

Two rows can have the same numeric shape and still inhabit a misleading representation. Hour 23 and hour 0 are 23 units apart when encoded as ordinary numbers, although they are adjacent on a clock. A circular sine-and-cosine encoding preserves that neighborhood. This is not cosmetic preprocessing: it replaces a line with a circle, the variable’s actual geometry.

Categories need similar care. Under one-hot encoding, any two distinct categories sit at square root of two from each other. That assigns the same separation to “red” and “blue” as to “red” and “ambulance.” It may be reasonable when identity alone matters, but it does not express semantic relations by itself. Missing values, binary indicators and redundant coordinates introduce comparable choices.

The accurate claim is therefore conditional: Euclidean distance measures straight-line length between vectors expressed in the same coordinate system. Geometric proximity is not useful similarity when units are incompatible, dimensions are excessive or the representation loses meaning. It can assign an observation to its nearest neighbor or centroid after features are transformed consistently; it cannot prove that the transformation preserves what matters.

Euclidean, Manhattan and cosine answer different questions

Manhattan distance adds absolute differences. Euclidean distance squares differences before aggregating them and therefore penalizes one large deviation in a coordinate more strongly. Neither is universally superior. If movement is restricted to a grid, Manhattan may better reflect the route; if the geometry permits a straight line and scales are coherent, Euclidean distance may be the natural choice.

Cosine similarity shifts attention from magnitude to direction. Its official scikit-learn definition is the dot product divided by the product of the norms. Two proportional vectors point in the same direction even when one is much longer. This can help with profiles where relative composition matters, but it is a poor choice when absolute size contains the signal that must be preserved.

When both vectors have Euclidean norm one, the notions are connected: squared distance equals 2 minus 2 times cosine similarity. Ranking by smaller Euclidean distance then matches ranking by larger cosine similarity. The equivalence depends on normalization; magnitude returns outside it. This identity reveals comparisons that appear to pit metrics against each other but actually pit preprocessing choices against each other.

What changes as dimensions are added

Every coordinate contributes a nonnegative term to the squared sum. If many dimensions contain noise, all add small differences that can obscure the few informative dimensions. Pestov’s paper proves, under explicit geometric assumptions, that the dimensionality curse in certain similarity searches is related to concentration of measure. It does not say that every high-dimensional database fails. It explains why a neighborhood can cease to be selective in important classes of spaces.

The useful test is not repeating that “high dimension is bad.” Measure the distribution of distances: nearest neighbor, typical distances and faraway points for representative queries. If relative gaps narrow as features are added, the ranking becomes sensitive to noise, rounding or new samples. Remove groups of coordinates as well and observe whether the neighbors and the downstream task’s performance change.

Column count and intrinsic dimension are not identical. A thousand variables may describe a structure with only a few degrees of freedom; twenty independent variables can be harder. Dimensionality reduction may help when it preserves relevant signal, but a clean-looking chart is not proof. Validate the transformation through neighbors, decisions or predictions on data not used to fit it.

The metric changes the algorithm too

Nearest-neighbor methods use a distance to retrieve nearby examples and, for instance, predict a label. The official scikit-learn neighbors guide presents Euclidean distance as the most common choice, while explaining that the best value of k depends on the data: increasing it suppresses noise but blurs decision boundaries. It also warns that ties among identically distant neighbors can depend on training-data order.

Computational cost cannot be separated from geometry. Brute-force comparison of every pair grows with both samples and dimensions. Structures such as KD-trees reject regions through distance bounds and are often effective in low dimensions; the same guide notes that they may become inefficient as dimension grows. Selecting a metric means choosing what counts as similar and which indexes and shortcuts remain valid.

The implementation deserves a numerical test

A library need not calculate each subtraction literally. To exploit matrix operations, scikit-learn’s Euclidean-distance implementation uses an identity involving norms and dot products. It is efficient and supports sparse matrices, but the documentation warns about catastrophic cancellation and says the resulting matrix may not be exactly symmetric. A mathematical property can acquire tiny floating-point deviations without changing the definition.

Before trusting a pipeline, test known points: zero distance from every row to itself; symmetry within a tolerance; the 3-4-5 triangle; and a case where one unit changes. Repeat with real-world scale, extreme and missing values, and the numeric types used in production. If a minute difference changes a tie or threshold, the system needs an explicit rule, not more decimal places.

A seven-question audit of “closeness”

First: what does one row represent? Second: what does each coordinate mean, and in which unit? Third: which transformation makes the columns comparable, and from which data was it learned? Fourth: why does straight-line length, absolute sum or direction match the objective? Fifth: how do distances behave when features are added or removed? Sixth: does the retrieved neighbor improve the task on data not used to design the system? Seventh: what is the cost of choosing the wrong neighbor?

These questions separate correct mathematics from a useful decision. A reproducible calculation can faithfully answer the wrong representation; a sensible representation can fail when the population shifts; and useful retrieval can feed a threshold with poorly chosen costs. The record should contain data version, preprocessing, metric, parameters, tie handling, distance distribution and downstream outcome, not only the final score.

The transferable skill is to read any claim of “similarity” as a verifiable chain: representation, scale, geometry, algorithm and decision. Euclidean distance is a precise rule, not a certificate of resemblance. When someone says two cases are close, you can now request the coordinates, change one unit, test another metric and see whether the nearest neighbor remains the same. That is where evidence begins.