The Demographic and Political Composition of Mechanical Turk Samples

Kevin E. Levay, Jeremy Freese, James N. Druckman*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

299 Scopus citations

Abstract

One of the most notable recent developments in survey research is the increased usage of online convenience samples drawn from Amazon’s Mechanical Turk (MTurk). While scholars have noted various social and political differences (e.g., age, partisanship) between MTurk and population-based samples, the breadth and depth of these variations remain unclear. We investigate the extent to which MTurk samples differ from population samples, and the underlying nature of these differences. We do so by replicating items from the population-based American National Election Studies (ANES) 2012 Time Series Study in a survey administered to a sample of MTurk respondents. With few exceptions, we not only find that MTurk respondents differ significantly from respondents completing the 2012 ANES via the Web but also that most differences are reduced considerably when controlling for easily measurable sample features. Thus, MTurk respondents do not appear to differ fundamentally from population-based respondents in unmeasurable ways. This suggests that MTurk data can be used to advance research programs, particularly if researchers measure and account for a range of political and demographic variables as needed.

Original languageEnglish (US)
JournalSAGE Open
Volume6
Issue number1
DOIs
StatePublished - Mar 4 2016

Keywords

  • Amazon’s Mechanical Turk
  • public opinion
  • research methods
  • survey sampling

ASJC Scopus subject areas

  • Arts and Humanities(all)
  • Social Sciences(all)

Fingerprint

Dive into the research topics of 'The Demographic and Political Composition of Mechanical Turk Samples'. Together they form a unique fingerprint.

Cite this