Maxi-Min discriminant analysis via online learning.

Journal Article (Journal Article)

Linear Discriminant Analysis (LDA) is an important dimensionality reduction algorithm, but its performance is usually limited on multi-class data. Such limitation is incurred by the fact that LDA actually maximizes the average divergence among classes, whereby similar classes with smaller divergence tend to be merged in the subspace. To address this problem, we propose a novel dimensionality reduction method called Maxi-Min Discriminant Analysis (MMDA). In contrast to the traditional LDA, MMDA attempts to find a low-dimensional subspace by maximizing the minimal (worst-case) divergence among classes. This "minimal" setting overcomes the problem of LDA that tends to merge similar classes with smaller divergence when used for multi-class data. We formulate MMDA as a convex problem and further as a large-margin learning problem. One key contribution is that we design an efficient online learning algorithm to solve the involved problem, making the proposed method applicable to large scale data. Experimental results on various datasets demonstrate the efficiency and the efficacy of our proposed method against five other competitive approaches, and the scalability to the data with thousands of classes.

Full Text

Duke Authors

Cited Authors

  • Xu, B; Huang, K; Liu, C-L

Published Date

  • October 2012

Published In

Volume / Issue

  • 34 /

Start / End Page

  • 56 - 64

PubMed ID

  • 22831850

Electronic International Standard Serial Number (EISSN)

  • 1879-2782

International Standard Serial Number (ISSN)

  • 0893-6080

Digital Object Identifier (DOI)

  • 10.1016/j.neunet.2012.06.001

Language

  • eng