Skip to main content

Learning overcomplete latent variable models through tensor methods

Publication ,  Conference
Anandkumar, A; Ge, R; Janzamin, M
Published in: Journal of Machine Learning Research
January 1, 2015

We provide guarantees for learning latent variable models emphasizing on the overcomplete regime, where the dimensionality of the latent space exceeds the observed dimensionality. In particular, we consider multiview mixtures, ICA, and sparse coding models. Our main tool is a new algorithm for tensor decomposition that works in the overcomplete regime. In the semi-supervised setting, we exploit label information to get a rough estimate of the model parameters, and then refine it using the tensor method on unlabeled samples. We establish learning guarantees when the number of components scales as k = o(dp/2), where d is the observed dimension, and p is the order of the observed moment employed in the tensor method (usually p = 3; 4). In the unsupervised setting, a simple initialization algorithm based on SVD of the tensor slices is proposed, and the guarantees are provided under the stricter condition that k ≤βd (where constant β can be larger than 1). For the learning applications, we provide tight sample complexity bounds through novel covering arguments.

Duke Scholars

Published In

Journal of Machine Learning Research

EISSN

1533-7928

ISSN

1532-4435

Publication Date

January 1, 2015

Volume

40

Issue

2015

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4905 Statistics
  • 4611 Machine learning
  • 17 Psychology and Cognitive Sciences
  • 08 Information and Computing Sciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Anandkumar, A., Ge, R., & Janzamin, M. (2015). Learning overcomplete latent variable models through tensor methods. In Journal of Machine Learning Research (Vol. 40).
Anandkumar, A., R. Ge, and M. Janzamin. “Learning overcomplete latent variable models through tensor methods.” In Journal of Machine Learning Research, Vol. 40, 2015.
Anandkumar A, Ge R, Janzamin M. Learning overcomplete latent variable models through tensor methods. In: Journal of Machine Learning Research. 2015.
Anandkumar, A., et al. “Learning overcomplete latent variable models through tensor methods.” Journal of Machine Learning Research, vol. 40, no. 2015, 2015.
Anandkumar A, Ge R, Janzamin M. Learning overcomplete latent variable models through tensor methods. Journal of Machine Learning Research. 2015.

Published In

Journal of Machine Learning Research

EISSN

1533-7928

ISSN

1532-4435

Publication Date

January 1, 2015

Volume

40

Issue

2015

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4905 Statistics
  • 4611 Machine learning
  • 17 Psychology and Cognitive Sciences
  • 08 Information and Computing Sciences