Abstract
It is known that for a certain class of single index models (SIMs) Y = f(Xpx1Tβ0, ε), support recovery is impossible when X ∼ script N (0, IIpxp) and a model complexity adjusted sample size is below a critical threshold. Recently, optimal algorithms based on Sliced Inverse Regression (SIR) were suggested. These algorithms work provably under the assumption that the design X comes from an i.i.d. Gaussian distribution. In the present paper we analyze algorithms based on covariance screening and least squares with L1 penalization (i.e. LASSO) and demonstrate that they can also enjoy optimal (up to a scalar) rescaled sample size in terms of support recovery, albeit under slightly different assumptions on f and ε compared to the SIR based algorithms. Furthermore, we show more generally, that LASSO succeeds in recovering the signed support of β0 if X ∼ script N (0, Σ), and the covariance Σ satisfies the irrepresentable condition. Our work extends existing results on the support recovery of LASSO for the linear model, to a more general class of SIMs.
| Original language | English (US) |
|---|---|
| Journal | Journal of Machine Learning Research |
| Volume | 17 |
| State | Published - May 1 2016 |
Funding
The authors would like to thank the Associate Editor and anonymous reviewers for their insightful remarks which led to the improvement of this manuscript. We are also grateful to Professor Noureddine El Karoui, who among other observations, brought to our attention that outcome transformations may be performed to facilitate the usage of LASSO even when c0 = 0 for the original outcome Y . This research was partially supported by Research Grants NSF DMS1208771, NIH R01 GM113242-01, NIH U54 HG007963 and NIH RO1 HL089778.
Keywords
- High-dimensional statistics
- LASSO
- Single index models
- Sparsity
- Support recovery
ASJC Scopus subject areas
- Software
- Artificial Intelligence
- Control and Systems Engineering
- Statistics and Probability