Skip to main content

Analysis of boosting algorithms using the smooth margin function

Publication ,  Journal Article
Rudin, C; Schapire, RE; Daubechies, I
Published in: Annals of Statistics
December 1, 2007

We introduce a useful tool for analyzing boosting algorithms called the "smooth margin function," a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, "coordinate ascent boosting" and "approximate coordinate ascent boosting," which are similar to Freund and Schapire's AdaBoost algorithm and Breiman's arc-gv algorithm. We give convergence rates to the maximum margin solution for both of our algorithms and for arc-gv. We then study AdaBoost's convergence properties using the smooth margin function. We precisely bound the margin attained by AdaBoost when the edges of the weak classifiers fall within a specified range. This shows that a previous bound proved by Ratsch and Warmuth is exactly tight. Furthermore, we use the smooth margin to capture explicit properties of AdaBoost in cases where cyclic behavior occurs. © Institute of Mathematical Statistics, 2007.

Duke Scholars

Published In

Annals of Statistics

DOI

ISSN

0090-5364

Publication Date

December 1, 2007

Volume

35

Issue

6

Start / End Page

2723 / 2768

Related Subject Headings

  • Statistics & Probability
  • 4905 Statistics
  • 3802 Econometrics
  • 1403 Econometrics
  • 0104 Statistics
  • 0102 Applied Mathematics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Rudin, C., Schapire, R. E., & Daubechies, I. (2007). Analysis of boosting algorithms using the smooth margin function. Annals of Statistics, 35(6), 2723–2768. https://doi.org/10.1214/009053607000000785
Rudin, C., R. E. Schapire, and I. Daubechies. “Analysis of boosting algorithms using the smooth margin function.” Annals of Statistics 35, no. 6 (December 1, 2007): 2723–68. https://doi.org/10.1214/009053607000000785.
Rudin C, Schapire RE, Daubechies I. Analysis of boosting algorithms using the smooth margin function. Annals of Statistics. 2007 Dec 1;35(6):2723–68.
Rudin, C., et al. “Analysis of boosting algorithms using the smooth margin function.” Annals of Statistics, vol. 35, no. 6, Dec. 2007, pp. 2723–68. Scopus, doi:10.1214/009053607000000785.
Rudin C, Schapire RE, Daubechies I. Analysis of boosting algorithms using the smooth margin function. Annals of Statistics. 2007 Dec 1;35(6):2723–2768.

Published In

Annals of Statistics

DOI

ISSN

0090-5364

Publication Date

December 1, 2007

Volume

35

Issue

6

Start / End Page

2723 / 2768

Related Subject Headings

  • Statistics & Probability
  • 4905 Statistics
  • 3802 Econometrics
  • 1403 Econometrics
  • 0104 Statistics
  • 0102 Applied Mathematics