A video is a sequence of images, but understanding it is more than running an image model on every frame. The extra ingredient is time: what moves, where it goes, and what is happening across many frames.
Optical flow estimates how each part of the image moves between two frames. Object tracking follows detected objects over time and keeps their identities consistent, which is the heart of multi-object tracking systems such as DeepSORT. Action recognition classifies what is happening in a clip, using models such as 3D CNNs that apply convolution across time as well as space.
The same ideas carry into more recent video architectures, including two-stream networks, SlowFast, and attention-based video models. Watch the lessons in order, then continue on to 3D Vision.
The Lucas-Kanade approach to optical flow, a method for estimating how image content moves between frames.
Multi-object tracking by detection: how DeepSORT combines a detector with a learned appearance embedder to keep object identities across frames.
An introduction to recognizing actions in video with 3D convolutional networks, which convolve across time as well as space.
A university lecture on video models: single-frame baselines, 3D CNNs, optical flow and two-stream networks, self-attention, and architectures such as C3D, I3D, and SlowFast.