Object Tracking
Following the same object across video frames — where a 200-line algorithm from 2016 still beats most deep learning.
When not to use it
- Before fixing your detector. Tracking-by-detection can't track what wasn't detected. Most tracking bugs are detection bugs.
- A deep tracker, before trying SORT. It's 200 lines and often enough.
- Kalman-only, through long occlusions. Motion prediction dies when the object is hidden. You need appearance.
- Single-camera methods across cameras. That's re-identification, and it's a much harder problem.
Reach for something else instead
- SORT — start here. Fast, simple, competitive.
- ByteTrack — keeps low-confidence detections. Simple and strong.
- DeepSORT — appearance matching for occlusion.
- Detection only — if you don't need identity across frames, you don't need this.
Sources & further reading
- Bewley et al. (2016), Simple Online and Realtime Tracking — SORT; 200 lines, Kalman plus Hungarian, still competitive.
- Wojke, Bewley & Paulus (2017), Simple Online and Realtime Tracking with a Deep Association Metric — DeepSORT; appearance for occlusion.
- Zhang et al. (2022), ByteTrack: Multi-Object Tracking by Associating Every Detection Box — keep the low-confidence boxes; they're your occluded objects.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Tuning the tracker when the detector is the ceiling.
- Discarding low-confidence detections. Those are your occluded objects — that's ByteTrack's whole insight.
- Expecting Kalman prediction to survive a long occlusion. Constant velocity through a wall is not a model.
- Reading SORT's survival as anti-learning. The learned detector does the heavy lifting; SORT is the well-specified part on top.
At a glance
FieldComputer Vision
The dominant approachtracking-by-detection: detect every frame, link the boxes
The algorithm that still winsSORT; Kalman filter + Hungarian algorithm, ~200 lines, no learning
Your ceilingthe detector
ByteTrack's insightlow-confidence detections are your occluded objects
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
SORT vs. an end-to-end tracking transformer — one is 200 lines of 1950s mathematics on top of a modern detector; the other is elegant and doesn't clearly beat it.