A bayesian approach to unsupervised feature selection and density estimation using expectation propagation


Journal Article

We propose an approximate Bayesian approach for unsupervised feature selection and density estimation, where the importance of the features for clustering is used as the measure for feature selection. Traditional maximum-likelihood (ML) model-parameter optimization schemes estimate the feature saliencies for a fixed model structure (i.e., a fixed number of clusters). In practice, the number of clusters present in the data for mixture-based modeling is unknown. In an ML framework, the number of clusters typically needs to be ascertained prior to estimating the feature saliencies. We propose a density estimation scheme that addresses model complexity (number of clusters present) and model-parameter estimation (feature saliencies) in a single optimization framework. The approximate Bayesian approach presented here, based on the expectation propagation method, obtains a full posterior distribution on the saliency of the features, along with full posterior distribution of other model parameters (including the number of clusters) that represent the underlying statistics of the data. The performance of the algorithm, is analyzed based on its ability to identify the features salient for clustering the multivariate data.

Full Text

Duke Authors

Cited Authors

  • Chang, S; Dasgupta, N; Carin, L

Published Date

  • January 1, 2005

Published In

  • Proceedings 2005 Ieee Computer Society Conference on Computer Vision and Pattern Recognition, Cvpr 2005

Volume / Issue

  • II /

Start / End Page

  • 1043 - 1050

Digital Object Identifier (DOI)

  • 10.1109/CVPR.2005.15

Citation Source

  • Scopus