Multi-View Learning

Multi-view learning is the family of methods that learns from several distinct feature sets —the «views»— that describe the same data from complementary angles. A web page, for instance, can be described by the text it contains and, separately, by the words in the hyperlinks that point to it; a video, by its image and its audio track. Each view is a full representation of the same object, not a fragment of it.

The intuition is simple: if the views carry partially independent information, combining them pays off more than using either alone and, crucially, it lets a model exploit unlabeled data —which is plentiful— instead of relying solely on hand-annotated examples.

Why it helps: views that complement each other

The approach rests on two ideas the literature calls consensus and complementarity. Under complementarity, each view supplies signal the other cannot see: a page's own text and its incoming links capture different facets of the same topic. Under consensus, the models trained on each view are pushed to agree on their predictions, and enforcing that agreement over unlabeled examples acts as extra guidance that lowers error. Two partially independent windows onto the same phenomenon constrain what each can conclude, and the system learns from material that a single feature set would have thrown away.

Blum and Mitchell's co-training (1998) and CCA

The foundational method is co-training, introduced by Avrim Blum and Tom Mitchell in 1998 to classify web pages. The idea: train two classifiers, one per view, on whatever few labeled examples exist; then let each classifier label the unlabeled cases it is most confident about and hand them to the other as fresh training data. The models teach each other, and the labeled set grows with no human in the loop. Related families follow from the same idea: co-regularization, which instead of trading labels adds a term that penalizes disagreement between views, and canonical correlation analysis (CCA), which dates back to Hotelling (1936) and seeks projections of two views that are maximally correlated, capturing the structure they share.

Assumptions, limits, and the bridge to multimodal learning

Classic co-training demands two stiff conditions, and it is worth being honest about them: each view must be sufficient on its own to predict the class, and the views must be conditionally independent given that class. Real data rarely satisfy either fully —a page's text and its links do, in fact, share a great deal of information— so the theoretical guarantees loosen and performance hinges on how genuinely complementary the views are. Multi-view learning is also the conceptual ancestor of today's multimodal learning, which fuses text, image, and audio; the difference is that the multi-view tradition usually worked with several feature sets of the same nature, whereas multimodal work crosses distinct sensory modalities. Through its use of unlabeled data, the approach is a close relative of semi-supervised learning too. What remains unsettled, in general, is how to tell in advance whether two views are independent enough for their combination to pay off: it stays an empirical question rather than a closed recipe.