t-SNE

t-SNE (t-distributed Stochastic Neighbor Embedding) is a nonlinear dimensionality-reduction technique whose main use is to visualize high-dimensional data, placing each point on a map of two or three dimensions. Its virtue is preserving above all the local structure: points that are close in the original space stay close in the map. Laurens van der Maaten and Geoffrey Hinton proposed it in 2008, as an improvement on an earlier technique, SNE.

How it works

The idea, in broad strokes, is to translate the similarities between pairs of points into probabilities: two nearby points have a high probability of being “neighbors.” t-SNE models those probabilities in the original space and looks for a two-dimensional arrangement whose probabilities resemble them as closely as possible. The use of a heavy-tailed Student's t distribution in the map mitigates the “crowding problem,” preventing all the points from piling up in the center.

Perplexity

Its key hyperparameter is perplexity, which regulates the balance between attending to local or global structure; in practice it moves within a modest range of values (typically between 5 and 50). It is advisable to try several and compare.

How to read (and not read) its maps

Here is the important part, because it is easy to over-interpret. t-SNE is stochastic: two runs give different maps. And, above all, the sizes of the groups and the distances between them are not reliable: the algorithm inflates dense clusters and shrinks sparse ones, and the separation between two groups on the map may mean nothing. With low perplexity, even noise can look like structure.

Its limits and alternatives

t-SNE is slow on very large sets and is designed to visualize, not as a feature-extraction step for other models. Unlike t-SNE, PCA is a linear transformation: it projects data onto axes ordered by the variance they explain, providing a useful reference for summarizing global structure without modeling nonlinear neighborhoods. UMAP, presented in 2018, was described by its authors as competitive with t-SNE in visualization quality, with superior runtime performance and, in their own wording, arguably better preservation of global structure.