Online Crowd Learning with Heterogeneous Workers via Majority Voting

Chao Huang, Haoran Yu, Jianwei Huang, Randall A. Berry

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many platforms recruit workers through crowd-sourcing to finish online tasks involving a huge amount of effort (e.g., image labeling and content moderation). These platforms aim to incentivize heterogeneous workers to exert effort finishing the tasks and truthfully report their solutions. When the verification for the workers' solutions is absent, the crowdsourcing problem is challenging and is known as information elicitation without verification (IEWV). Majority voting is a common approach to solve an IEWV problem, where a worker is rewarded based on whether his solution is consistent with the majority. However, most prior related work relies on a strong assumption that workers' solution accuracy levels are public knowledge. We relax such an assumption and propose an online learning mechanism based on majority voting, which allows the platform to learn the distribution of the workers' solution accuracy levels. In the mechanism, workers will be asked to report their private accuracy levels (which do not need to be the true values), in addition to deciding their effort levels and solution reporting strategies. The mechanism computes the workers' rewards based on their reported accuracy levels, and the workers obtain rewards if their reported solutions match the majority. We show that our mechanism induces workers to truthfully report their solution accuracy levels in the long run, in which the platform asymptotically achieves zero regret. Moreover, we show that our online mechanism converges faster when the workers are more capable of solving the tasks.

Original languageEnglish (US)
Title of host publication2020 18th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOPT 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9783903176294
StatePublished - Jun 2020
Event18th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOPT 2020 - Volos, Greece
Duration: Jun 15 2020Jun 19 2020

Publication series

Name2020 18th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOPT 2020

Conference

Conference18th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, WiOPT 2020
CountryGreece
CityVolos
Period6/15/206/19/20

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems and Management
  • Control and Optimization
  • Modeling and Simulation

Fingerprint Dive into the research topics of 'Online Crowd Learning with Heterogeneous Workers via Majority Voting'. Together they form a unique fingerprint.

Cite this