Online Crowd Learning Through Strategic Worker Reports

Chao Huang, Haoran Yu, Jianwei Huang*, Randall A. Berry

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


When it is difficult to verify contributed solutions in mobile crowdsourcing, the majority voting mechanism is widely utilized to incentivize distributed workers to provide high-quality and truthful solutions. In the majority voting mechanism, a worker is rewarded based on whether his solution is consistent with the majority. However, most prior related work relies on a strong assumption that workers' solution accuracy levels are public knowledge, which may not hold in many practical scenarios. We relax such an assumption and propose an online mechanism, which allows the platform to learn the distribution of the workers' solution accuracy levels via asking workers to report their private accuracy levels (which do not need to be the true values), in addition to deciding their effort levels and solution reporting strategies. The mechanism design is challenging, as neither the workers' task solutions nor their accuracy reports can be verified. We devise a randomized reward mechanism that computes the workers' rewards based on their reported accuracy levels, under which the workers obtain rewards if their reported solutions match the majority. We show that our mechanism induces workers to truthfully report their solution accuracy levels in the long run, in which the empirical accuracy distribution (collected from workers' accuracy reports) converges to the actual accuracy distribution. Moreover, we show that our online mechanism converges faster when the workers are more capable of solving the tasks.

Original languageEnglish (US)
Pages (from-to)5406-5417
Number of pages12
JournalIEEE Transactions on Mobile Computing
Issue number9
StatePublished - Sep 1 2023


  • Mobile crowdsourcing
  • game theory
  • incentive mechanism design
  • online learning

ASJC Scopus subject areas

  • Software
  • Electrical and Electronic Engineering
  • Computer Networks and Communications


Dive into the research topics of 'Online Crowd Learning Through Strategic Worker Reports'. Together they form a unique fingerprint.

Cite this