Decision-making under uncertainty in research synthesis: Designing for the garden of forking paths

Alex Kale, Matthew Kay, Jessica Ruth Hullman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

To make evidence-based recommendations to decision-makers, researchers conducting systematic reviews and meta-analyses must navigate a garden of forking paths: a series of analytical decision-points, each of which has the potential to influence findings. To identify challenges and opportunities related to designing systems to help researchers manage uncertainty around which of multiple analyses is best, we interviewed 11 professional researchers who conduct research synthesis to inform decision-making within three organizations. We conducted a qualitative analysis identifying 480 analytical decisions made by researchers throughout the scientific process. We present descriptions of current practices in applied research synthesis and corresponding design challenges: making it more feasible for researchers to try and compare analyses, shifting researchers' attention from rationales for decisions to impacts on results, and supporting communication techniques that acknowledge decision-makers' aversions to uncertainly. We identify opportunities to design systems which help researchers explore, reason about, and communicate uncertainly in decision-making about possible analyses in research synthesis.

Original languageEnglish (US)
Title of host publicationCHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450359702
DOIs
StatePublished - May 2 2019
Event2019 CHI Conference on Human Factors in Computing Systems, CHI 2019 - Glasgow, United Kingdom
Duration: May 4 2019May 9 2019

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2019 CHI Conference on Human Factors in Computing Systems, CHI 2019
CountryUnited Kingdom
CityGlasgow
Period5/4/195/9/19

Keywords

  • Representing uncertainty
  • Research synthesis

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Decision-making under uncertainty in research synthesis: Designing for the garden of forking paths'. Together they form a unique fingerprint.

Cite this