This article needs to be updated.(April 2023) |
This article may be too technical for most readers to understand.(September 2023) |
Part of a series on |
Machine learning and data mining |
---|
In machine learning, boosting is an ensemble meta-algorithm for primarily reducing bias, variance.[1] It is used in supervised learning and a family of machine learning algorithms that convert weak learners to strong ones.[2]
The concept of boosting is based on the question posed by Kearns and Valiant (1988, 1989):[3][4] "Can a set of weak learners create a single strong learner?" A weak learner is defined to be a classifier that is only slightly correlated with the true classification (it can label examples better than random guessing). In contrast, a strong learner is a classifier that is arbitrarily well-correlated with the true classification.
Robert Schapire answered the question posed by Kearns and Valiant in the affirmative in a paper published in 1990[5].This has had significant ramifications in machine learning and statistics, most notably leading to the development of boosting.[6]
When first introduced, the hypothesis boosting problem simply referred to the process of turning a weak learner into a strong learner. "Informally, [the hypothesis boosting] problem asks whether an efficient learning algorithm […] that outputs a hypothesis whose performance is only slightly better than random guessing [i.e. a weak learner] implies the existence of an efficient algorithm that outputs a hypothesis of arbitrary accuracy [i.e. a strong learner]."[3] Algorithms that achieve hypothesis boosting quickly became simply known as "boosting". Freund and Schapire's arcing (Adapt[at]ive Resampling and Combining),[7] as a general technique, is more or less synonymous with boosting.[8]
Arcing [Boosting] is more successful than bagging in variance reduction
The term boosting refers to a family of algorithms that are able to convert weak learners to strong learners
Schapire (1990) proved that boosting is possible. (Page 823)