Pose Estimation

Pose estimation is the computer vision task of locating a person's body keypoints—joints such as the shoulders, elbows, hips, and knees—in images and video, and using them to reconstruct a skeleton that describes posture and movement. The term also covers object pose: estimating the rotation and translation with which a rigid object sits in front of the camera.

Two families are worth telling apart. Human pose estimation locates people's joints and can extend to the feet, hands, and face, as OpenPose does. Object pose estimation recovers a rigid body's six degrees of freedom (6DoF)—three of rotation, three of translation—a core problem in robotics and augmented reality, tracked by benchmarks such as BOP.

2D or 3D, Top-Down or Bottom-Up

2D estimation places keypoints in image coordinates; the de facto standard is the 17-point set annotated in the COCO dataset. 3D estimation reconstructs their positions in space, and its reference benchmark is Human3.6M, with 3.6 million poses recorded through motion capture. With several people in the scene, two strategies compete: top-down methods first detect each person and then estimate their keypoints, while bottom-up methods, such as OpenPose with its Part Affinity Fields (PAFs), detect every keypoint in the image and then group them by individual.

How It Is Measured

In 2D, the PCK metric (percentage of correct keypoints) counts a point as correct if it falls within a fraction of a reference size, such as the head in its PCKh variant. COCO uses OKS (Object Keypoint Similarity), which compares prediction and annotation through a Gaussian over the distance between them, normalized by the person's scale and a per-keypoint constant, and computes average precision (AP) on top of it. In 3D, MPJPE dominates: the mean Euclidean distance, in millimeters, between predicted and ground-truth joints; leading systems report errors below 45 millimeters on Human3.6M.

Current Models and Tools

OpenPose (Cao et al., Carnegie Mellon University) was the first open-source real-time system for multi-person 2D pose estimation. HRNet (CVPR 2019) raised accuracy by keeping high-resolution representations in parallel throughout the network. Google's MediaPipe Pose brings the task to phones: its BlazePose model estimates 33 landmarks with both image and world coordinates directly on the device. RTMPose, from OpenMMLab's MMPose ecosystem, is an industry-oriented top-down method that reaches 75.8% AP on COCO at more than 90 frames per second on a consumer CPU, and Ultralytics' YOLO family offers pose variants that detect COCO's 17 keypoints in real time.

Uses and Risks

Pose estimation underpins biomechanical analysis in sports, physiotherapy and remote rehabilitation, markerless motion capture for animation, and the robotic manipulation and augmented reality that depend on objects' 6DoF pose. Its technical limits persist: occlusions, loose clothing, and crowded scenes still degrade results.

The risks are documented too. Pose feeds gait recognition: Chinese company Watrix claimed in 2018 that it could identify a person from 50 meters away, even from behind or with the face covered, and the Associated Press and the South China Morning Post reported on its trials with police in Beijing and Shanghai. Researchers also warn that training datasets represent body types, ages, and contexts unevenly, a bias whose real-world extent still lacks systematic measurement. In Europe, the AI Act bans real-time remote biometric identification in publicly accessible spaces for law enforcement, save for narrowly defined exceptions.