Contrastive learning from pairwise measurements

Yi Chen, Zhuoran Yang, Yuchen Xie, Zhaoran Wang

Research output: Contribution to journalConference article

Abstract

Learning from pairwise measurements naturally arises from many applications, such as rank aggregation, ordinal embedding, and crowdsourcing. However, most existing models and algorithms are susceptible to potential model misspecification. In this paper, we study a semiparametric model where the pairwise measurements follow a natural exponential family distribution with an unknown base measure. Such a semiparametric model includes various popular parametric models, such as the Bradley-Terry-Luce model and the paired cardinal model, as special cases. To estimate this semiparametric model without specifying the base measure, we propose a data augmentation technique to create virtual examples, which enables us to define a contrastive estimator. In particular, we prove that such a contrastive estimator is invariant to model misspecification within the natural exponential family, and moreover, attains the optimal statistical rate of convergence up to a logarithmic factor. We provide numerical experiments to corroborate our theory.

Original languageEnglish (US)
Pages (from-to)10909-10918
Number of pages10
JournalAdvances in Neural Information Processing Systems
Volume2018-December
StatePublished - Jan 1 2018
Event32nd Conference on Neural Information Processing Systems, NeurIPS 2018 - Montreal, Canada
Duration: Dec 2 2018Dec 8 2018

Fingerprint

Agglomeration
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Cite this

Chen, Yi ; Yang, Zhuoran ; Xie, Yuchen ; Wang, Zhaoran. / Contrastive learning from pairwise measurements. In: Advances in Neural Information Processing Systems. 2018 ; Vol. 2018-December. pp. 10909-10918.
@article{c5654b2963f54346a010ae8e70c17966,
title = "Contrastive learning from pairwise measurements",
abstract = "Learning from pairwise measurements naturally arises from many applications, such as rank aggregation, ordinal embedding, and crowdsourcing. However, most existing models and algorithms are susceptible to potential model misspecification. In this paper, we study a semiparametric model where the pairwise measurements follow a natural exponential family distribution with an unknown base measure. Such a semiparametric model includes various popular parametric models, such as the Bradley-Terry-Luce model and the paired cardinal model, as special cases. To estimate this semiparametric model without specifying the base measure, we propose a data augmentation technique to create virtual examples, which enables us to define a contrastive estimator. In particular, we prove that such a contrastive estimator is invariant to model misspecification within the natural exponential family, and moreover, attains the optimal statistical rate of convergence up to a logarithmic factor. We provide numerical experiments to corroborate our theory.",
author = "Yi Chen and Zhuoran Yang and Yuchen Xie and Zhaoran Wang",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
volume = "2018-December",
pages = "10909--10918",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",

}

Chen, Y, Yang, Z, Xie, Y & Wang, Z 2018, 'Contrastive learning from pairwise measurements', Advances in Neural Information Processing Systems, vol. 2018-December, pp. 10909-10918.

Contrastive learning from pairwise measurements. / Chen, Yi; Yang, Zhuoran; Xie, Yuchen; Wang, Zhaoran.

In: Advances in Neural Information Processing Systems, Vol. 2018-December, 01.01.2018, p. 10909-10918.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Contrastive learning from pairwise measurements

AU - Chen, Yi

AU - Yang, Zhuoran

AU - Xie, Yuchen

AU - Wang, Zhaoran

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Learning from pairwise measurements naturally arises from many applications, such as rank aggregation, ordinal embedding, and crowdsourcing. However, most existing models and algorithms are susceptible to potential model misspecification. In this paper, we study a semiparametric model where the pairwise measurements follow a natural exponential family distribution with an unknown base measure. Such a semiparametric model includes various popular parametric models, such as the Bradley-Terry-Luce model and the paired cardinal model, as special cases. To estimate this semiparametric model without specifying the base measure, we propose a data augmentation technique to create virtual examples, which enables us to define a contrastive estimator. In particular, we prove that such a contrastive estimator is invariant to model misspecification within the natural exponential family, and moreover, attains the optimal statistical rate of convergence up to a logarithmic factor. We provide numerical experiments to corroborate our theory.

AB - Learning from pairwise measurements naturally arises from many applications, such as rank aggregation, ordinal embedding, and crowdsourcing. However, most existing models and algorithms are susceptible to potential model misspecification. In this paper, we study a semiparametric model where the pairwise measurements follow a natural exponential family distribution with an unknown base measure. Such a semiparametric model includes various popular parametric models, such as the Bradley-Terry-Luce model and the paired cardinal model, as special cases. To estimate this semiparametric model without specifying the base measure, we propose a data augmentation technique to create virtual examples, which enables us to define a contrastive estimator. In particular, we prove that such a contrastive estimator is invariant to model misspecification within the natural exponential family, and moreover, attains the optimal statistical rate of convergence up to a logarithmic factor. We provide numerical experiments to corroborate our theory.

UR - http://www.scopus.com/inward/record.url?scp=85064816959&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064816959&partnerID=8YFLogxK

M3 - Conference article

VL - 2018-December

SP - 10909

EP - 10918

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -