Pose Estimation
Finding the joints of a body in an image — solved well enough to be boring, and the applications are mostly about watching people.
When not to use it
- As anonymous data. Gait is identifying. People are recognisable from how they move, at distance, without a face.
- Through occlusion. Crossed arms and crowds produce confident nonsense.
- For 3D, from one camera, expecting accuracy. The depth ambiguity is the same projective problem.
- On atypical bodies. The learned prior is "people bend like this," and gymnasts, dancers and wheelchair users break it.
Reach for something else instead
- Marker-based mocap — accurate, expensive, studio-bound.
- IMU sensors — wearables. No cameras, no occlusion.
- Depth cameras — resolves the 3D ambiguity with a measurement.
- Object detection — if you only need to know a person is there, don't take their skeleton.
Sources & further reading
- Cao et al. (2019), OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields — the method that made multi-person real-time work.
- Loper et al. (2015), SMPL: A Skinned Multi-Person Linear Model — the parametric body model everything 3D uses.
- Andriluka et al. (2014), 2D Human Pose Estimation: New Benchmark and State of the Art Analysis — MPII; the benchmark that drove the field.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Claiming skeletons are anonymous. Gait recognition exists; that claim is doing a lot of work in surveillance contracts.
- Regressing joint coordinates directly. Heatmaps are better — spatial output for a spatial problem.
- Expecting 3D lifting to be reliable. Infinitely many 3D poses project to the same 2D skeleton.
- Ignoring who's in the training data. Less studied than face recognition, same shape of problem.
At a glance
FieldComputer Vision
What it givesa skeleton: 17-25 joints, real-time, multi-person
The elegant bitPart Affinity Fields make crowd assignment polynomial
The privacy claim"it's just a skeleton" is doing a lot of work; gait is identifying
Where it livesa regulatory gap its growth market depends on
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Pose estimation vs. face recognition — one tells you who someone is and is regulated; the other tells you what they're doing and mostly isn't.