Tree-based inference for dirichlet process mixtures
The Dirichlet process mixture (DPM) is a widely used model for clustering and for nonparametric Bayesian density estimation. Unfortunately, like in many statistical models, exact inference in a DPM is intractable, and approximate methods are needed to perform efficient inference. While most attention has been placed on Markov chain Monte Carlo (MCMC) (Escobar and West, 1995; Neal, 2000; Rasmussen, 2000), variational Bayesian (VB) (Blei and Jordan, 2005) and collapsed variational methods (Kurihara, Welling and Teh, 2007), Heller and Ghahramani (2005) recently introduced a new class of approximation for DPMs based on Bayesian hierarchical clustering (BHC). These tree-based combinatorial approximations efficiently sum over exponentially many ways of partitioning the data and offer a novel lower bound on the marginal likelihood of DPMs. In this paper we make the following contributions: (1) We show empirically that the BHC lower bounds are substantially tighter than the bounds given by VB and by collapsed variational methods on synthetic and real datasets. (2) We show that BHC offers a better predictive performance on these datasets. (3) We improve the tree-based lower bounds with an algorithm that efficiently sums contributions from alternative trees. (4) We present a fast approximate method for BHC. Our results suggest that our approximate inference methods and lower bounds may be useful not only in DPMs but in other models as well. © 2009 by the authors.
Xu, Y; Heller, KA; Ghahramani, Z
Volume / Issue
Start / End Page
Electronic International Standard Serial Number (EISSN)
International Standard Serial Number (ISSN)