Computer Vision

Computer vision is the field of artificial intelligence that seeks to give machines the ability to interpret and extract information from the visual world: images and video. It is not about capturing pixels —any camera does that— but about understanding what they contain: which objects appear, where they are, how they move and what they mean in context.

Its goal is to turn a grid of color values into useful, actionable descriptions. Where the human eye effortlessly recognizes a face, a road sign or an abnormal cell, a machine must learn to do the same from data —and that learning now powers much of applied AI.

The tasks it solves

The field is organized around a set of canonical tasks. Image classification assigns a single label to a whole image; object detection locates and labels multiple items with bounding boxes; segmentation —semantic or instance— labels the image pixel by pixel. To these we can add recognition of faces and text (OCR), pose estimation, which infers the position of a body, and tracking of objects across a video. Several have their own entry in this glossary —see object detection and segmentation— and nearly all of them now rest on convolutional neural networks (CNNs), covered separately. Primary source. Primary source. Primary source. Primary source.

The arc: from SIFT and HOG to multimodal models

For decades, seeing by computer meant hand-designing the relevant features: descriptors such as David Lowe's SIFT or HOG (histogram of oriented gradients) captured edges and textures to feed classical classifiers. In 2012, AlexNet —the deep convolutional network by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton— won ILSVRC with a 15.3% top-5 error rate, versus 26.2% for the runner-up. That result quantifies the leap; “launched an era” is a historical interpretation, not a measurement in the paper. Deeper, more stable architectures followed, such as ResNet and its residual connections. In October 2020, Alexey Dosovitskiy and colleagues introduced the Vision Transformer (ViT): a pure Transformer applied directly to sequences of image patches, bringing to vision an architecture already standard in natural-language processing. Among foundation and multimodal models, CLIP learned visual representations by predicting which text matched each image and enabled zero-shot transfer. SAM was trained to respond to prompts and transfer without task-specific tuning to new segmentation distributions and tasks; “zero-shot” describes that transfer, not an absence of training: its SA-1B dataset contains over one billion masks on eleven million images. Primary source.

Uses and limits

Computer vision already underpins self-driving cars, medical imaging diagnosis, industrial quality control and biometrics. Yet it is far from a solved problem. Models remain brittle against adversarial examples —tiny perturbations, invisible to us, that fool them completely—, inherit biases from their training data, and lose robustness when a scene drifts from what they saw in training: different lighting, a new angle, another population. These advances do not establish general visual reliability: CLIP's authors reported that zero-shot transfer was often only competitive with a ResNet-50 baseline already well below the overall state of the art, and documented sensitivity to how classes are worded; SAM, for its part, reports results on specific segmentation tasks. The evidence remains specific to a task, dataset, and metric—not equivalent to a universal comparison with human vision.