An ensemble combines many simple models into one stronger model. A single decision tree can overfit, but a group of trees that vote together is usually far more accurate and stable. Ensembles power many of the best-performing methods on tabular data.
You'll start with random forests, which build many trees on random samples of the data and average their predictions. Then you'll move to boosting, where models are built one after another, each learning from the mistakes of the last: first AdaBoost, then gradient boosting, and finally XGBoost, a fast and widely used implementation.
Watch the videos in order, then try a random forest and a gradient boosting model on a dataset of your own and compare them to a single tree. When you're done, continue on to k-NN, SVM & Naive Bayes.
How a random forest builds many trees from bootstrapped data and random feature subsets, and how out-of-bag samples estimate its accuracy.
How AdaBoost combines weak learners called stumps, gives each an amount of say, and reweights the samples it gets wrong.
How gradient boosting builds trees on the residuals of earlier trees, using a learning rate to improve predictions step by step.
How XGBoost builds regularized trees using similarity scores and gain, and how pruning and the learning rate keep it from overfitting.