Enriching Meta-Analytic Models of Summary Data: A Thought Experiment and Case Study

Blakeley B. McShane*, Ulf Böckenholt

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Meta-analysis typically involves the analysis of summary data (e.g., means, standard deviations, and sample sizes) from a set of studies via a statistical model that is a special case of a hierarchical (or multilevel) model. Unfortunately, the common summary-data approach to meta-analysis used in psychological research is often employed in settings where the complexity of the data warrants alternative approaches. In this article, we propose a thought experiment that can lead meta-analysts to move away from the common summary-data approach to meta-analysis and toward richer and more appropriate summary-data approaches when the complexity of the data warrants it. Specifically, we propose that it can be extremely fruitful for meta-analysts to act as if they possess the individual-level data from the studies and consider what model specifications they might fit even when they possess only summary data. This thought experiment is justified because (a) the analysis of the individual-level data from the studies via a hierarchical model is considered the “gold standard” for meta-analysis and (b) for a wide variety of cases common in meta-analysis, the summary-data and individual-level-data approaches are, by a principle known as statistical sufficiency, equivalent when the underlying models are appropriately specified. We illustrate the value of our thought experiment via a case study that evolves across five parts that cover a wide variety of data settings common in meta-analysis.

Original languageEnglish (US)
Pages (from-to)81-93
Number of pages13
JournalAdvances in Methods and Practices in Psychological Science
Volume3
Issue number1
DOIs
StatePublished - Mar 2020

Keywords

  • Between-study variation
  • Heterogeneity
  • Hierarchical model
  • Meta-analysis
  • Multilevel model
  • Open data
  • Open materials
  • Random effects

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint

Dive into the research topics of 'Enriching Meta-Analytic Models of Summary Data: A Thought Experiment and Case Study'. Together they form a unique fingerprint.

Cite this