Average Power: A Cautionary Note

Blakeley B. McShane*, Ulf Böckenholt, Karsten T. Hansen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Replication is an important contemporary issue in psychological research, and there is great interest in ways of assessing replicability, in particular, retrospectively via prior studies. The average power of a set of prior studies is a quantity that has attracted considerable attention for this purpose, and techniques to estimate this quantity via a meta-analytic approach have recently been proposed. In this article, we have two aims. First, we clarify the nature of average power and its implications for replicability. We explain that average power is not relevant to the replicability of actual prospective replication studies. Instead, it relates to efforts in the history of science to catalogue the power of prior studies. Second, we evaluate the statistical properties of point estimates and interval estimates of average power obtained via the meta-analytic approach. We find that point estimates of average power are too variable and inaccurate for use in application. We also find that the width of interval estimates of average power depends on the corresponding point estimates; consequently, the width of an interval estimate of average power cannot serve as an independent measure of the precision of the point estimate. Our findings resolve a seeming puzzle posed by three estimates of the average power of the power-posing literature obtained via the meta-analytic approach.

Original languageEnglish (US)
Pages (from-to)185-199
Number of pages15
JournalAdvances in Methods and Practices in Psychological Science
Volume3
Issue number2
DOIs
StatePublished - Jun 2020

Keywords

  • effect size
  • history of science
  • meta-analysis
  • open materials
  • power
  • publication bias

ASJC Scopus subject areas

  • General Psychology

Fingerprint

Dive into the research topics of 'Average Power: A Cautionary Note'. Together they form a unique fingerprint.

Cite this