Meta unveils V-JEPA, a model that learns from video without pixels

Meta has unveiled V-JEPA, an artificial intelligence model trained to learn from videos without human labels and without trying to reconstruct every missing pixel in a sequence. Instead, it predicts an internal, abstract representation of the hidden parts of the video.

The approach matters because it moves away from the dominant method of generating images or frames. For Yann LeCun, Meta’s chief AI scientist, understanding the world requires systems to learn which elements of a scene matter, how they change and what might happen next—not merely produce a visually convincing copy.

Predicting meaning, not exact appearance

V-JEPA stands for Video Joint Embedding Predictive Architecture. Its operation is based on a simple idea: areas of a clip are hidden, and the model must infer what is there from the rest of the sequence.

But it does not return the missing pixels. V-JEPA generates a representation—a kind of high-level mathematical summary—of what should be in that area. That representation can capture the fact that a person is lifting an object or that a vehicle is moving, without having to determine the exact color of a shirt, the background or the texture of a wall.

The model has one encoder that analyzes the visible parts of the video and another that produces the target representations for the hidden regions. A predictor learns to approximate those representations from the available context. During training, the system receives videos without annotations of their objects or actions: this is self-supervised learning because the data itself supplies the learning task.

This difference has practical consequences. Predicting pixels forces a system to devote computing power to details that may be irrelevant to understanding an action. In addition, when several visual continuations are possible, a generative model may tend to produce blurry or artificial results. An abstract representation can preserve the information needed to reason about the scene without requiring a single exact future image.

A piece of LeCun’s bet on world models

V-JEPA embodies a line of research that LeCun has defended for years: machines should build world models—systems with an internal understanding of objects, actions and basic causal relationships.

Large language models learn primarily from text and excel at continuing sequences of words. Generative image and video models, meanwhile, have shown that they can create increasingly realistic audiovisual content. Meta’s thesis is that generating content and understanding a situation are not necessarily the same task.

A child does not need to imagine every pixel of a ball behind a sofa to anticipate that it may reappear on the other side. It is enough to retain an understanding that the ball still exists, along with its trajectory and the obstacles in the way. V-JEPA aims to help a visual system learn abstractions of this kind from large collections of video.

The approach also connects with I-JEPA, the image model Meta introduced in 2023. The new element is time: video contains movements, changes and relationships between consecutive moments that a still image cannot show.

Strong results in action understanding

Meta trained V-JEPA on approximately two million unlabeled public videos and evaluated it on action-classification and action-localization tasks. According to the company, the model delivered competitive results on benchmark datasets such as Kinetics-400, Something-Something-v2 and AVA, even when the learned encoder was kept frozen and only an additional layer was trained for the specific task.

That last test is significant. If a representation works across multiple tasks without having to retrain the entire model, the cost of adapting it to new uses may fall. In theory, this kind of technology could help analyze activities in video, find specific moments in large audiovisual archives or give robotic systems a better understanding of what is happening around them.

Even so, recognizing actions on benchmarks is not the same as understanding the world as a person does. Evaluation datasets tend to have defined categories and relatively constrained videos; the real world is more ambiguous, shifts with context and contains unexpected situations.

The challenge is moving from observation to anticipation

V-JEPA is not a video assistant or a consumer-ready product. It is open research into an alternative way of training visual systems. Its value lies in suggesting that the next advance does not have to be a more spectacular video generator, but rather a model that discards superficial information and retains what enables it to anticipate a scene.

Meta will have to show that the idea scales to longer sequences, less controlled environments and tasks where predictions have real-world consequences. It also remains to be seen how these visual representations can be combined with language, memory and planning. But V-JEPA makes one divide clear—one that will be central in the years ahead: creating convincing images does not guarantee an understanding of what is happening in them.