Meta-analysis of Studies with Multiple Contrasts and Differences in Measurement Scales

The common approach to meta-analysis is overwhelmingly dominant in practice but suffers from a major limitation: It is suitable for analyzing only a single effect of interest. However, contemporary psychological research studies—and thus meta-analyses of them—typically feature multiple dependent effects of interest. In this paper, we introduce novel meta-analytic methodology that (a) accommodates an arbitrary number of effects—specifically, contrasts of means—and (b) yields results in standard deviation units in order to adjust for differences in the measurement scales used for the dependent measure across studies. Importantly, when all studies follow the same two-condition study design and interest centers on the simple contrast between the two conditions as measured on the standardized mean difference (or Cohen’s d) scale, our approach is equivalent to the common approach. Consequently, our approach generalizes the common approach to accommodate an arbitrary number of contrasts. As we illustrate and elaborate on across three extensive case studies, our approach has several advantages relative to the common approach. To facilitate the use of our approach, we provide a website that implements it.