A network starts with random weights, so its first predictions are poor. To improve, it needs to know two things: how wrong it is and which way to change each weight to be less wrong. A loss measures the error, and gradient descent repeatedly nudges every weight and bias a small step in the direction that lowers it.
The hard part is finding that direction for millions of weights at once. Backpropagation solves it by applying the chain rule layer by layer, passing the error backwards from the output to the input so every weight gets its own gradient. Together, a forward pass, a backward pass and a weight update make one step of the training loop.
Watch the lessons in order. The third video covers the calculus and can be skipped on a first pass, and the Karpathy lecture is a longer coding walkthrough. Then continue to Loss Functions & Optimizers.
3Blue1Brown on what it means for a network to learn: a cost function that measures error, and gradient descent as the method that nudges the weights and biases downhill to reduce it.
Video 20min3Blue1Brown gives an intuitive picture of backpropagation, the algorithm that works out how each weight and bias should change, by pushing the error backwards through the network.
Video 12minThe math behind backpropagation: how the chain rule produces the derivatives that gradient descent needs. Optional if you only want the intuition, useful if you want the formulas.
Video 10minAndrej Karpathy builds a tiny autograd engine from scratch: derivatives and the chain rule, forward and backward passes, and then trains a small neural network with gradient descent. A long, hands-on watch.
Video 2h 25min