Skip to main content

Adaptive Markov chain Monte Carlo for Bayesian variable selection

Publication ,  Journal Article
Ji, C; Schmidler, SC
Published in: Journal of Computational and Graphical Statistics
December 17, 2013

We describe adaptive Markov chain Monte Carlo (MCMC) methods for sampling posterior distributions arising from Bayesian variable selection problems. Point-mass mixture priors are commonly used in Bayesian variable selection problems in regression. However, for generalized linear and nonlinear models where the conditional densities cannot be obtained directly, the resulting mixture posterior may be difficult to sample using standard MCMC methods due to multimodality. We introduce an adaptive MCMC scheme that automatically tunes the parameters of a family of mixture proposal distributions during simulation. The resulting chain adapts to sample efficiently from multimodal target distributions. For variable selection problems point-mass components are included in the mixture, and the associated weights adapt to approximate marginal posterior variable inclusion probabilities, while the remaining components approximate the posterior over nonzero values. The resulting sampler transitions efficiently between models, performing parameter estimation and variable selection simultaneously. Ergodicity and convergence are guaranteed by limiting the adaptation based on recent theoretical results. The algorithm is demonstrated on a logistic regression model, a sparse kernel regression, and a random field model from statistical biophysics; in each case the adaptive algorithm dramatically outperforms traditional MH algorithms. Supplementary materials for this article are available online. © 2013 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.

Duke Scholars

Published In

Journal of Computational and Graphical Statistics

DOI

EISSN

1537-2715

ISSN

1061-8600

Publication Date

December 17, 2013

Volume

22

Issue

3

Start / End Page

708 / 728

Related Subject Headings

  • Statistics & Probability
  • 4905 Statistics
  • 1403 Econometrics
  • 0104 Statistics
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Ji, C., & Schmidler, S. C. (2013). Adaptive Markov chain Monte Carlo for Bayesian variable selection. Journal of Computational and Graphical Statistics, 22(3), 708–728. https://doi.org/10.1080/10618600.2013.819178
Ji, C., and S. C. Schmidler. “Adaptive Markov chain Monte Carlo for Bayesian variable selection.” Journal of Computational and Graphical Statistics 22, no. 3 (December 17, 2013): 708–28. https://doi.org/10.1080/10618600.2013.819178.
Ji C, Schmidler SC. Adaptive Markov chain Monte Carlo for Bayesian variable selection. Journal of Computational and Graphical Statistics. 2013 Dec 17;22(3):708–28.
Ji, C., and S. C. Schmidler. “Adaptive Markov chain Monte Carlo for Bayesian variable selection.” Journal of Computational and Graphical Statistics, vol. 22, no. 3, Dec. 2013, pp. 708–28. Scopus, doi:10.1080/10618600.2013.819178.
Ji C, Schmidler SC. Adaptive Markov chain Monte Carlo for Bayesian variable selection. Journal of Computational and Graphical Statistics. 2013 Dec 17;22(3):708–728.

Published In

Journal of Computational and Graphical Statistics

DOI

EISSN

1537-2715

ISSN

1061-8600

Publication Date

December 17, 2013

Volume

22

Issue

3

Start / End Page

708 / 728

Related Subject Headings

  • Statistics & Probability
  • 4905 Statistics
  • 1403 Econometrics
  • 0104 Statistics