FastText

FastText is an open-source library for building vector representations of words (word embeddings) and for classifying text, developed by Facebook AI Research (now Meta AI). It is lightweight, fast and runs on ordinary hardware. Its contribution was not inventing embeddings —word2vec and GloVe had already popularised them— but looking inside the word: instead of treating each term as an indivisible unit, it breaks it into smaller pieces.

That seemingly minor decision solves two problems that dogged earlier models: words the system has never seen, and languages with rich morphology.

The core idea: a word is a bag of character n-grams

In «Enriching Word Vectors with Subword Information» (TACL, 2017), Piotr Bojanowski, Édouard Grave, Armand Joulin and Tomáš Mikolov propose representing each word as a bag of character n-grams —the chunks of letters that make it up. The word «where», for instance, breaks down into pieces such as «whe», «her», «ere» and the whole word itself. Each of those pieces gets its own vector, and a word's vector is the sum of its n-gram vectors. The model inherits word2vec's skipgram scheme but operates over subwords rather than whole words.

Why it handles words it has never seen

This is where FastText's most-cited advantage comes from. A classic model that only knows whole words is stumped by any term missing from its training vocabulary —the out-of-vocabulary (OOV) problem. FastText is not: even if it has never encountered «retweeting», it can assemble the word's vector by summing n-grams it already knows from other words. That makes it especially effective for morphologically rich languages —Turkish, Finnish or German, where a single root spawns dozens of derived forms— and for neologisms, typos and proper nouns. Shared fragments between related words («run», «running», «runner») pull their vectors together without the model having to see every form.

Text classification and where it fits

FastText also ships a text classifier, introduced in «Bag of Tricks for Efficient Text Classification» (Joulin, Grave, Bojanowski and Mikolov, 2016). Its promise is speed without sacrificing accuracy: it trains on billions of words in minutes on an ordinary CPU and rivals far more expensive deep-learning classifiers. To stay fast when there are many categories, it uses a hierarchical softmax. Tasks such as sentiment analysis or topic tagging fit squarely into this mode.

Historically, FastText was a bridge: it enriched the static embeddings of word2vec and GloVe with subword information and helped pave the way for the contextual embeddings of the transformers that followed. It remains a valuable tool whenever speed, a small footprint and open vocabularies matter.