Recommender Systems

A recommender system is an information-filtering system that predicts a user's preference for an item and suggests the most relevant ones: products, films, songs, news. They are the silent engine of much of the internet: what appears on your home page of a store, a video service or a music platform is largely decided by a recommender.

The approaches

There are three families. Collaborative filtering recommends from the behavior of many users (“people similar to you also liked this”). Content-based filtering recommends items similar to those you already liked, according to their attributes. And hybrid systems combine both; comparative studies show they tend to give more accurate recommendations than either of the pure approaches.

What data they use

Recommenders feed on two types of signal. Explicit ones, when the user rates something openly (a star rating), and implicit ones, inferred from their behavior: clicks, purchases, viewing time. Implicit signals are more abundant and less intrusive to collect, but also more ambiguous.

Their problems

Several challenges are cross-cutting. The cold start: a new user or item, with no history, is hard to recommend. Sparsity and scalability when the catalog is huge. And the filter bubble: if the system only reinforces what you already like, it impoverishes discovery; hence the interest in diversity and serendipity.

Evolution and metrics

The technique has evolved from neighborhood methods to matrix factorization—which was cemented by the Netflix Prize—and from there to deep learning and to sequential recommenders that model the order of interactions. To evaluate them, prediction-error metrics such as RMSE are used, and ranking metrics that measure the quality of the offered list, such as precision@k or NDCG.