Weak Classifier

Question

I am trying to implement an application that uses AdaBoost algorithm. I know that AdaBoost uses set of weak classifiers, but I don't know what these weak classifiers are. Can you explain it to me with an example and tell me if I have to create my own weak classifiers or I'm suppoused to use some kind of algorithm?

Answer 1

When I used AdaBoost, my weak classifiers were basically thresholds for each data attribute. Those thresholds need to have a performance of more than 50%, if not it would be totally random.

Here is a good presentation about Adaboost and how to calculate those weak classifiers: http://www.cse.cuhk.edu.hk/~lyu/seminar/07spring/Hongbo.ppt

Answer 2

Weak classifiers (or weak learners) are classifiers which perform only slightly better than a random classifier. These are thus classifiers which have some clue on how to predict the right labels, but not as much as strong classifiers have like, e.g., Naive Bayes, Neurel Networks or SVM.

One of the simplest weak classifiers is the Decision Stump, which is a one-level Decision Tree. It selects a threshold for one feature and splits the data on that threshold. AdaBoost will then train an army of these Decision Stumps which each focus on one part of the characteristics of the data.