Week 3 - Ensemble Bagging and Boosting
OMSCS
The idea of Ensemble learning is to look at a subset of data and try to model that well. If we do that to enough small patches we hope we can combine this to give us a good overall understanding of the data.
This is a powerful idea when the function we are trying to map doesn’t globally generalise but instead locally generalises like when people try to filter out spam from your inbox.
Bagging
The simplest example of Ensemble learning is Bagging.
Bagging treated all data points equally and didn’t focus on whether we performed well or poorly on a given data point to pick the next subset. If we fixed this we could potentially tighten up our model.
Error rate and weak learners
Models that are considered good should always do better than chance.