TY - JOUR
T1 - Enriching Meta-Analytic Models of Summary Data
T2 - A Thought Experiment and Case Study
AU - McShane, Blakeley B.
AU - Böckenholt, Ulf
N1 - Funding Information:
This article is dedicated to Murray McShane.
Publisher Copyright:
© 2020, SAGE Publications Inc.. All rights reserved.
PY - 2020/3
Y1 - 2020/3
N2 - Meta-analysis typically involves the analysis of summary data (e.g., means, standard deviations, and sample sizes) from a set of studies via a statistical model that is a special case of a hierarchical (or multilevel) model. Unfortunately, the common summary-data approach to meta-analysis used in psychological research is often employed in settings where the complexity of the data warrants alternative approaches. In this article, we propose a thought experiment that can lead meta-analysts to move away from the common summary-data approach to meta-analysis and toward richer and more appropriate summary-data approaches when the complexity of the data warrants it. Specifically, we propose that it can be extremely fruitful for meta-analysts to act as if they possess the individual-level data from the studies and consider what model specifications they might fit even when they possess only summary data. This thought experiment is justified because (a) the analysis of the individual-level data from the studies via a hierarchical model is considered the “gold standard” for meta-analysis and (b) for a wide variety of cases common in meta-analysis, the summary-data and individual-level-data approaches are, by a principle known as statistical sufficiency, equivalent when the underlying models are appropriately specified. We illustrate the value of our thought experiment via a case study that evolves across five parts that cover a wide variety of data settings common in meta-analysis.
AB - Meta-analysis typically involves the analysis of summary data (e.g., means, standard deviations, and sample sizes) from a set of studies via a statistical model that is a special case of a hierarchical (or multilevel) model. Unfortunately, the common summary-data approach to meta-analysis used in psychological research is often employed in settings where the complexity of the data warrants alternative approaches. In this article, we propose a thought experiment that can lead meta-analysts to move away from the common summary-data approach to meta-analysis and toward richer and more appropriate summary-data approaches when the complexity of the data warrants it. Specifically, we propose that it can be extremely fruitful for meta-analysts to act as if they possess the individual-level data from the studies and consider what model specifications they might fit even when they possess only summary data. This thought experiment is justified because (a) the analysis of the individual-level data from the studies via a hierarchical model is considered the “gold standard” for meta-analysis and (b) for a wide variety of cases common in meta-analysis, the summary-data and individual-level-data approaches are, by a principle known as statistical sufficiency, equivalent when the underlying models are appropriately specified. We illustrate the value of our thought experiment via a case study that evolves across five parts that cover a wide variety of data settings common in meta-analysis.
KW - Between-study variation
KW - Heterogeneity
KW - Hierarchical model
KW - Meta-analysis
KW - Multilevel model
KW - Open data
KW - Open materials
KW - Random effects
UR - http://www.scopus.com/inward/record.url?scp=85102676497&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85102676497&partnerID=8YFLogxK
U2 - 10.1177/2515245919884304
DO - 10.1177/2515245919884304
M3 - Article
AN - SCOPUS:85102676497
VL - 3
SP - 81
EP - 93
JO - Advances in Methods and Practices in Psychological Science
JF - Advances in Methods and Practices in Psychological Science
SN - 2515-2459
IS - 1
ER -