Skip to main content

Boosting based on a smooth margin

Publication ,  Journal Article
Rudin, C; Schapire, RE; Daubechies, I
Published in: Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)
January 1, 2004

We study two boosting algorithms, Coordinate Ascent Boosting and Approximate Coordinate Ascent Boosting, which are explicitly designed to produce maximum margins. To derive these algorithms, we introduce a smooth approximation of the margin that one can maximize in order to produce a maximum margin classifier. Our first algorithm is simply coordinate ascent on this function, involving a line search at each step. We then make a simple approximation of this line search to reveal our second algorithm. These algorithms are proven to asymptotically achieve maximum margins, and we provide two convergence rate calculations. The second calculation yields a faster rate of convergence than the first, although the first gives a more explicit (still fast) rate. These algorithms are very similar to AdaBoost in that they are based on coordinate ascent, easy to implement, and empirically tend to converge faster than other boosting algorithms. Finally, we attempt to understand AdaBoost in terms of our smooth margin, focusing on cases where AdaBoost exhibits cyclic behavior.

Duke Scholars

Published In

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

DOI

ISSN

0302-9743

Publication Date

January 1, 2004

Volume

3120

Start / End Page

502 / 517

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 46 Information and computing sciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Rudin, C., Schapire, R. E., & Daubechies, I. (2004). Boosting based on a smooth margin. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), 3120, 502–517. https://doi.org/10.1007/978-3-540-27819-1_35
Rudin, C., R. E. Schapire, and I. Daubechies. “Boosting based on a smooth margin.” Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science) 3120 (January 1, 2004): 502–17. https://doi.org/10.1007/978-3-540-27819-1_35.
Rudin C, Schapire RE, Daubechies I. Boosting based on a smooth margin. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). 2004 Jan 1;3120:502–17.
Rudin, C., et al. “Boosting based on a smooth margin.” Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), vol. 3120, Jan. 2004, pp. 502–17. Scopus, doi:10.1007/978-3-540-27819-1_35.
Rudin C, Schapire RE, Daubechies I. Boosting based on a smooth margin. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). 2004 Jan 1;3120:502–517.

Published In

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

DOI

ISSN

0302-9743

Publication Date

January 1, 2004

Volume

3120

Start / End Page

502 / 517

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 46 Information and computing sciences