Optimal linear estimation under unknown nonlinear transform

Xinyang Yi, Zhaoran Wang, Constantine Caramanis, Han Liu

Research output: Contribution to journalConference article

12 Scopus citations

Abstract

Linear regression studies the problem of estimating a model parameter β ∈ ℝ, from n observations {(yi,xi)} n i=1 from linear model yi = (x i β ) + ∈ i . We consider a significant generalization in which the relationship between (x i , β } and yi is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing. We propose a novel spectral-based estimation procedure and show that we can recover β in settings (i.e., classes of link function f) where previous algorithms fail. In general, our algorithm requires only very mild restrictions on the (unknown) functional relationship between yi and <x i , β >. We also consider the high dimensional setting where β is sparse, and introduce a two-stage nonconvex framework that addresses estimation challenges in high dimensional regimes where p n. For a broad class of link functions between (x i , β } and y i , we establish minimax lower bounds that demonstrate the optimality of our estimators in both the classical and high dimensional regimes.

Original languageEnglish (US)
Pages (from-to)1549-1557
Number of pages9
JournalAdvances in Neural Information Processing Systems
Volume2015-January
StatePublished - Jan 1 2015
Event29th Annual Conference on Neural Information Processing Systems, NIPS 2015 - Montreal, Canada
Duration: Dec 7 2015Dec 12 2015

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint Dive into the research topics of 'Optimal linear estimation under unknown nonlinear transform'. Together they form a unique fingerprint.

  • Cite this