Skip to main content

Sparse linear identifiable multivariate modeling

Publication ,  Journal Article
Henao, R; Winther, O
Published in: Journal of Machine Learning Research
March 1, 2011

In this paper we consider sparse and identifiable linear latent variable (factor) and linear Bayesian network models for parsimonious analysis of multivariate data. We propose a computationally efficient method for joint parameter and model inference, and model comparison. It consists of a fully Bayesian hierarchy for sparse models using slab and spike priors (two-component δ-function and continuous mixtures), non-Gaussian latent factors and a stochastic search over the ordering of the variables. The framework, which we call SLIM (Sparse Linear Identifiable Multivariate modeling), is validated and bench-marked on artificial and real biological data sets. SLIM is closest in spirit to LiNGAM (Shimizu et al., 2006), but differs substantially in inference, Bayesian network structure learning and model comparison. Experimentally, SLIM performs equally well or better than LiNGAM with comparable computational complexity. We attribute this mainly to the stochastic search strategy used, and to parsimony (sparsity and identifiability), which is an explicit part of the model. We propose two extensions to the basic i.i.d. linear framework: non-linear dependence on observed variables, called SNIM (Sparse Non-linear Identifiable Multivariate modeling) and allowing for correlations between latent variables, called CSLIM (Correlated SLIM), for the temporal and/or spatial data. The source code and scripts are available from http://cogsys.imm.dtu.dk/ slim/. © 2011 Ricardo Henao and Ole Winther.

Duke Scholars

Published In

Journal of Machine Learning Research

EISSN

1533-7928

ISSN

1532-4435

Publication Date

March 1, 2011

Volume

12

Start / End Page

863 / 905

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4905 Statistics
  • 4611 Machine learning
  • 17 Psychology and Cognitive Sciences
  • 08 Information and Computing Sciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Henao, R., & Winther, O. (2011). Sparse linear identifiable multivariate modeling. Journal of Machine Learning Research, 12, 863–905.
Henao, R., and O. Winther. “Sparse linear identifiable multivariate modeling.” Journal of Machine Learning Research 12 (March 1, 2011): 863–905.
Henao R, Winther O. Sparse linear identifiable multivariate modeling. Journal of Machine Learning Research. 2011 Mar 1;12:863–905.
Henao, R., and O. Winther. “Sparse linear identifiable multivariate modeling.” Journal of Machine Learning Research, vol. 12, Mar. 2011, pp. 863–905.
Henao R, Winther O. Sparse linear identifiable multivariate modeling. Journal of Machine Learning Research. 2011 Mar 1;12:863–905.

Published In

Journal of Machine Learning Research

EISSN

1533-7928

ISSN

1532-4435

Publication Date

March 1, 2011

Volume

12

Start / End Page

863 / 905

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4905 Statistics
  • 4611 Machine learning
  • 17 Psychology and Cognitive Sciences
  • 08 Information and Computing Sciences