Video Analysis

Video analysis applies computer vision to sequences of images ordered in time, not to an isolated image. Its defining feature versus still-image analysis is the temporal dimension: video encodes motion and change, so a model must answer not only “what is there,” but “what is happening” and “in what order.”

Tasks, not just applications

The formal tasks should be distinguished from their uses. The main ones are video classification or action recognition (assigning an action label to a clip), object detection in each frame, tracking (keeping an object's identity over time), segmentation (labeling pixel by pixel, in its semantic, instance and video variants) and temporal understanding (locating when an action begins and ends, summarizing or describing the video).

From 3D convolutions to transformers

Methods have evolved in generations. Early architectures combined convolution with the temporal dimension, such as I3D (DeepMind, 2017) or SlowFast (2019), with a slow pathway for appearance and a fast one for motion. Then came the video transformers, which apply the attention mechanism to space and time—ViViT and TimeSformer (2021), or VideoMAE (2022), with self-supervised pre-training. Today the multimodal video-language models dominate, coupling a video encoder with a large language model to describe and reason about what happens.

Evaluation and risks

Performance is measured on sets such as Kinetics or Something-Something, with accuracy, mean average precision or overlap metrics depending on the task. And there are risks worth naming. Demographic bias is real: the Gender Shades study (2018) found far higher error rates for dark-skinned women, and aggregate accuracy hides those per-subgroup differences. And use for surveillance—facial recognition and mass tracking—raises privacy concerns. These are technical and public-policy questions, which we report as such.