A model that scores well on the data it was trained on tells you very little. What matters is how it performs on data it has never seen. This lesson shows how to hold data back so you can measure that honestly, and how to avoid fooling yourself along the way.
You'll start with the basic train / validation / test split, then learn k-fold cross-validation, which rotates which part of the data is held out so every observation is used for both training and testing. You'll also see how to run it in Python with scikit-learn and how it helps you choose between models.
Watch the videos in order, then try running k-fold cross-validation on a dataset of your own. When you're done, continue on to Overfitting & Regularization.
Why we split data into training and testing sets, and how k-fold cross-validation lets you compare machine learning methods using all of your data.
A Python walkthrough with scikit-learn: why evaluating on training data misleads you, how a train/test split works, and how cross-validation gives a more reliable estimate.
How k-fold cross-validation splits a dataset into k folds to compare models, and how to run it with scikit-learn's cross_val_score.