Gibbs posterior for variable selection in high-dimensional classification and data mining

Wenxin Jiang*, Martin A. Tanner

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

38 Scopus citations

Abstract

In the popular approach of "Bayesian variable selection" (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is constructed from a risk function of practical interest (such as the classification error) and aims at minimizing a risk function without modeling the data probabilistically. This can improve the performance over the usual Bayesian approach, which depends on a probability model which may be misspecified. Conditions will be provided to achieve good risk performance, even in the presence of high dimensionality, when the number of candidate variables "K" can be much larger than the sample size "n." In addition, we develop a convenient Markov chain Monte Carlo algorithm to implement BVS with the Gibbs posterior.

Original languageEnglish (US)
Pages (from-to)2207-2231
Number of pages25
JournalAnnals of Statistics
Volume36
Issue number5
DOIs
StatePublished - Oct 2008

Keywords

  • Data augmentation
  • Data mining
  • Gibbs posterior
  • High-dimensional data
  • Linear classification
  • Markov chain Monte Carlo
  • Prior distribution
  • Risk performance
  • Sparsity
  • Variable selection

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint Dive into the research topics of 'Gibbs posterior for variable selection in high-dimensional classification and data mining'. Together they form a unique fingerprint.

Cite this