Training an AI model is an optimization problem: find the settings that make the error as small as possible. The gradient from the last module tells you which way is downhill, and gradient descent repeatedly takes small steps in that direction until the error stops falling.
The lessons start with gradient descent itself and the learning rate, which controls the step size and decides whether training is smooth, slow, or unstable. They then cover the improvements used in practice, such as momentum, RMSprop, and Adam, and finish with the shape of the problem itself: the difference between convex and non-convex landscapes.
Watch the lessons in order, then continue on to Probability Fundamentals.
A visual walkthrough of gradient descent that builds the algorithm from scratch in Python.
Video 28minA practical guide to the learning rate and how it affects gradient descent.
Video 4minExplains how the Adam optimizer builds on momentum and RMSProp.
Video 3minA tour of the optimizers used to train deep learning models: momentum, RMSprop, AdaGrad, and Adam.
Video 15min