More supervision, less computation: Statistical-computational tradeoffs in weakly supervised learning

Xinyang Yi, Zhaoran Wang, Zhuoran Yang, Constantine Caramanis, Han Liu

Research output: Contribution to journalConference article

Abstract

We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability 1 - α. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degree of supervision, which is quantified by α. In this paper, we characterize the effect of α by establishing the information-theoretic and computational boundaries, namely, the minimax-optimal statistical accuracy that can be achieved by all algorithms, and polynomial-time algorithms under an oracle computational model. For small α, our result shows a gap between these two boundaries, which represents the computational price of achieving the information-theoretic boundary due to the lack of supervision. Interestingly, we also show that this gap narrows as α increases. In other words, having more supervision, i.e., more correct labels, not only improves the optimal statistical accuracy as expected, but also enhances the computational efficiency for achieving such accuracy.

Original languageEnglish (US)
Pages (from-to)4482-4490
Number of pages9
JournalAdvances in Neural Information Processing Systems
StatePublished - Jan 1 2016
Event30th Annual Conference on Neural Information Processing Systems, NIPS 2016 - Barcelona, Spain
Duration: Dec 5 2016Dec 10 2016

Fingerprint

Supervised learning
Computational efficiency
Labels
Polynomials

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

@article{261f95ec6c1249fcb17ebaab8cff8bda,
title = "More supervision, less computation: Statistical-computational tradeoffs in weakly supervised learning",
abstract = "We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability 1 - α. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degree of supervision, which is quantified by α. In this paper, we characterize the effect of α by establishing the information-theoretic and computational boundaries, namely, the minimax-optimal statistical accuracy that can be achieved by all algorithms, and polynomial-time algorithms under an oracle computational model. For small α, our result shows a gap between these two boundaries, which represents the computational price of achieving the information-theoretic boundary due to the lack of supervision. Interestingly, we also show that this gap narrows as α increases. In other words, having more supervision, i.e., more correct labels, not only improves the optimal statistical accuracy as expected, but also enhances the computational efficiency for achieving such accuracy.",
author = "Xinyang Yi and Zhaoran Wang and Zhuoran Yang and Constantine Caramanis and Han Liu",
year = "2016",
month = "1",
day = "1",
language = "English (US)",
pages = "4482--4490",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",

}

More supervision, less computation : Statistical-computational tradeoffs in weakly supervised learning. / Yi, Xinyang; Wang, Zhaoran; Yang, Zhuoran; Caramanis, Constantine; Liu, Han.

In: Advances in Neural Information Processing Systems, 01.01.2016, p. 4482-4490.

Research output: Contribution to journalConference article

TY - JOUR

T1 - More supervision, less computation

T2 - Statistical-computational tradeoffs in weakly supervised learning

AU - Yi, Xinyang

AU - Wang, Zhaoran

AU - Yang, Zhuoran

AU - Caramanis, Constantine

AU - Liu, Han

PY - 2016/1/1

Y1 - 2016/1/1

N2 - We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability 1 - α. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degree of supervision, which is quantified by α. In this paper, we characterize the effect of α by establishing the information-theoretic and computational boundaries, namely, the minimax-optimal statistical accuracy that can be achieved by all algorithms, and polynomial-time algorithms under an oracle computational model. For small α, our result shows a gap between these two boundaries, which represents the computational price of achieving the information-theoretic boundary due to the lack of supervision. Interestingly, we also show that this gap narrows as α increases. In other words, having more supervision, i.e., more correct labels, not only improves the optimal statistical accuracy as expected, but also enhances the computational efficiency for achieving such accuracy.

AB - We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability 1 - α. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the statistical accuracies and computational efficiency of these algorithms depend on the degree of supervision, which is quantified by α. In this paper, we characterize the effect of α by establishing the information-theoretic and computational boundaries, namely, the minimax-optimal statistical accuracy that can be achieved by all algorithms, and polynomial-time algorithms under an oracle computational model. For small α, our result shows a gap between these two boundaries, which represents the computational price of achieving the information-theoretic boundary due to the lack of supervision. Interestingly, we also show that this gap narrows as α increases. In other words, having more supervision, i.e., more correct labels, not only improves the optimal statistical accuracy as expected, but also enhances the computational efficiency for achieving such accuracy.

UR - http://www.scopus.com/inward/record.url?scp=85019231342&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019231342&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85019231342

SP - 4482

EP - 4490

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -