Skip to main content
Journal cover image

On the dynamics of boosting

Publication ,  Conference
Rudin, C; Daubechies, I; Schapire, RE
Published in: Advances in Neural Information Processing Systems
January 1, 2004

In order to understand AdaBoost's dynamics, especially its ability to maximize margins, we derive an associated simplified nonlinear iterated map and analyze its behavior in low-dimensional cases. We find stable cycles for these cases, which can explicitly be used to solve for Ada- Boost's output. By considering AdaBoost as a dynamical system, we are able to prove R̈atsch and Warmuth's conjecture that AdaBoost may fail to converge to a maximal-margin combined classifier when given a 'nonoptimal' weak learning algorithm. AdaBoost is known to be a coordinate descent method, but other known algorithms that explicitly aim to maximize the margin (such as AdaBoost and arc-gv) are not. We consider a differentiable function for which coordinate ascent will yield a maximum margin solution. We then make a simple approximation to derive a new boosting algorithm whose updates are slightly more aggressive than those of arcgv.

Duke Scholars

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

ISBN

9780262201520

Publication Date

January 1, 2004

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Rudin, C., Daubechies, I., & Schapire, R. E. (2004). On the dynamics of boosting. In Advances in Neural Information Processing Systems.
Rudin, C., I. Daubechies, and R. E. Schapire. “On the dynamics of boosting.” In Advances in Neural Information Processing Systems, 2004.
Rudin C, Daubechies I, Schapire RE. On the dynamics of boosting. In: Advances in Neural Information Processing Systems. 2004.
Rudin, C., et al. “On the dynamics of boosting.” Advances in Neural Information Processing Systems, 2004.
Rudin C, Daubechies I, Schapire RE. On the dynamics of boosting. Advances in Neural Information Processing Systems. 2004.
Journal cover image

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

ISBN

9780262201520

Publication Date

January 1, 2004

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology