Jaccard Distance

The Jaccard index measures how similar two sets are: it is the size of their intersection divided by the size of their union. The Jaccard distance is its complement—one minus the index—and measures, conversely, how different they are. The index ranges from 0 (no elements in common) to 1 (identical sets).

The formula

Formally, the index is written J(A, B) = |A ∩ B| / |A ∪ B|: the shared elements over all the distinct elements. An example: if A = {apple, banana, orange, kiwi} and B = {banana, kiwi, pineapple}, they share 2 elements out of 5 distinct ones, so J = 2/5 = 0.4, and the distance is 1 − 0.4 = 0.6. The Jaccard distance is, moreover, a metric: it satisfies the triangle inequality.

Where it comes from

It was devised by the Swiss botanist Paul Jaccard, who published it in 1901—he called it the “coefficient of community”—to compare which plant species different zones of the Alps and the Jura shared. From ecology it jumped to statistics and computing. It has a close relative, the Sørensen-Dice coefficient, which counts the intersection twice over and gives somewhat higher values on the same data.

Its uses in AI

Jaccard similarity is a very versatile tool. It serves to compare documents and detect duplicates, representing each text as a set of fragments; to scale that comparison to millions of documents, the MinHash technique is used. And, very notably, in computer vision it reappears under another name: the intersection over union (IoU), which evaluates how much the box or mask predicted by an object detector overlaps with the true ones, is nothing other than the Jaccard index applied to regions of an image.