Process consistency for AdaBoost

Wenxin Jiang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

87 Scopus citations

Abstract

Recent experiments and theoretical studies show that AdaBoost can overfit in the limit of large time. If running the algorithm forever is suboptimal, a natural question is how low can the prediction error be during the process of AdaBoost? We show under general regularity conditions that during the process of AdaBoost a consistent prediction is generated, which has the prediction error approximating the optimal Bayes error as the sample size increases. This result suggests that, while running the algorithm forever can be suboptimal, it is reasonable to expect that some regularization method via truncation of the process may lead to a near-optimal performance for sufficiently large sample size.

Original languageEnglish (US)
Pages (from-to)13-29
Number of pages17
JournalAnnals of Statistics
Volume32
Issue number1
DOIs
StatePublished - Feb 2004

Keywords

  • AdaBoost
  • Bayes error
  • Boosting
  • Consistency
  • Prediction error
  • VC dimension

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint

Dive into the research topics of 'Process consistency for AdaBoost'. Together they form a unique fingerprint.

Cite this