A model can fit its training data almost perfectly and still make poor predictions on new data. This is overfitting, and it is one of the most common problems in machine learning. This lesson explains why it happens and what you can do about it.
You'll start with bias and variance, the two sources of error behind underfitting and overfitting. Then you'll learn regularization: ridge (L2) regression, lasso (L1) regression, and elastic net, which combines the two. You'll see how each one adds a penalty that makes a model less sensitive to its training data, and why lasso can set parameters to zero while ridge cannot.
Watch the videos in order, then try fitting ridge and lasso models and use cross-validation to choose the penalty strength. When you're done, continue on to Decision Trees.
The two key sources of prediction error, and how they relate to underfitting and overfitting.
How adding an L2 penalty makes a model less sensitive to the training data, and how to choose the penalty with cross-validation.
How lasso differs from ridge, and how it can shrink some parameters all the way to zero.
A visual explanation of why lasso can set parameters to zero while ridge cannot.
How elastic net combines the ridge and lasso penalties to get the benefits of both.