Deep mixtures of factor analyzers with common loadings: A novel deep generative approach to clustering

In this paper, we propose a novel deep density model, called Deep Mixtures of Factor Analyzers with Common Loadings (DMCFA). Employing a mixture of factor analyzers sharing common component loadings, this novel model is more physically meaningful, since the common loadings can be regarded as feature selection or reduction matrices. Importantly, the novel DMCFA model is able to remarkably reduce the number of free parameters, making the involved inferences and learning problem dramatically easier. Despite its simplicity, by engaging learnable Gaussian distributions as the priors, DMCFA does not sacrifice its flexibility in estimating the data density. This is particularly the case when compared with the existing model Deep Mixtures of Factor Analyzers (DMFA), exploiting different loading matrices but simple standard Gaussian distributions for each component prior. We evaluate the performance of the proposed DMCFA in comparison with three other competitive models including Mixtures of Factor Analyzers (MFA), MCFA, and DMFA and their shallow counterparts. Results on four real data sets show that the novel model demonstrates significantly better performance in both density estimation and clustering.

Duke Scholars

Author Kaizhu Huang DKU Faculty