Bayesian variable selection for high dimensional generalized linear models: Convergence rates of the fitted densities

Bayesian variable selection has gained much empirical success recently in a variety of applications when the number K of explanatory variables (x ₁, . . . , x_K) is possibly much larger than the sample size n. For generalized linear models, if most of the x_j's have very small effects on the response y, we show that it is possible to use Bayesian variable selection to reduce overfitting caused by the curse of dimensionality K ≫ n. In this approach a suitable prior can be used to choose a few out of the many x_j's to model y, so that the posterior will propose probability densities p that are "often close" to the true density p in some sense. The closeness can be described by a Hellinger distance between p and p that scales at a power very close to n^-1/2, which is the "finite-dimensional rate" corresponding to a low-dimensional situation. These findings extend some recent work of Jiang [Technical Report 05-02 (2005) Dept. Statistics, Northwestern Univ.] on consistency of Bayesian variable selection for binary classification.