Academics are more specific, and practitioners more sensitive, in forecasting interventions to strengthen democratic attitudes

James Y. Chu*, Jan G. Voelkel, Michael N. Stagnaro, Suji Kang, James N. Druckman, David G. Rand, Robb Willer

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Concern over democratic erosion has led to a proliferation of proposed interventions to strengthen democratic attitudes in the United States. Resource constraints, however, prevent implementing all proposed interventions. One approach to identify promising interventions entails leveraging domain experts, who have knowledge regarding a given field, to forecast the effectiveness of candidate interventions. We recruit experts who develop general knowledge about a social problem (academics), experts who directly intervene on the problem (practitioners), and nonexperts from the public to forecast the effectiveness of interventions to reduce partisan animosity, support for undemocratic practices, and support for partisan violence. Comparing 14,076 forecasts submitted by 1,181 forecasters against the results of a megaexperiment (n = 32,059) that tested 75 hypothesized effects of interventions, we find that both types of experts outperformed members of the public, though experts differed in how they were accurate. While academics' predictions were more specific (i.e., they identified a larger proportion of ineffective interventions and had fewer false-positive forecasts), practitioners' predictions were more sensitive (i.e., they identified a larger proportion of effective interventions and had fewer false-negative forecasts). Consistent with this, practitioners were better at predicting best-performing interventions, while academics were superior in predicting which interventions performed worst. Our paper highlights the importance of differentiating types of experts and types of accuracy. We conclude by discussing factors that affect whether sensitive or specific forecasters are preferable, such as the relative cost of false positives and negatives and the expected rate of intervention success.

Original languageEnglish (US)
Article numbere2307008121
JournalProceedings of the National Academy of Sciences of the United States of America
Volume121
Issue number3
DOIs
StatePublished - 2024
Externally publishedYes

Keywords

  • behavioral interventions
  • expertise
  • forecasting

ASJC Scopus subject areas

  • General

Fingerprint

Dive into the research topics of 'Academics are more specific, and practitioners more sensitive, in forecasting interventions to strengthen democratic attitudes'. Together they form a unique fingerprint.

Cite this