Abstract
Some experimental designs involve clustering within only one treatment group. Such designs may involve group tutoring, therapy administered by multiple therapists, or interventions administered by clinics for the treatment group, whereas the control group receives no treatment. In such cases, the data analysis often proceeds as if there were no clustering within the treatment group. A consequence is that the actual significance level of the treatment effects is larger (i.e., actual p values are larger) than nominal. Additionally, biases will be introduced in estimates of the effect sizes and their variances, leading to inflated effects and underestimated variances when clustering in the treatment group is not taken into account. These consequences of clustering can seriously compromise the interpretation of study results. This article shows how information on the intraclass correlation can be used to obtain a correction for biases in the effect sizes and their variances, and also to obtain an adjustment to the significance test for the effects of clustering.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 1295-1308 |
| Number of pages | 14 |
| Journal | Behavior Research Methods |
| Volume | 47 |
| Issue number | 4 |
| DOIs | |
| State | Published - Nov 26 2014 |
Funding
This article is based in part on work supported by the US Institute of Education Sciences (IES) under Grant Nos. R305D11032 and R305B1000027, and by the National Science Foundation (NSF) under Grant No. 0815295. Any opinions, findings, and conclusions or recommendations are those of the authors and do not necessarily represent the views of the IES or the NSF.
Keywords
- Effect size
- Meta-analysis
- Partial clustering
- Significance tests
ASJC Scopus subject areas
- Experimental and Cognitive Psychology
- Developmental and Educational Psychology
- Arts and Humanities (miscellaneous)
- Psychology (miscellaneous)
- General Psychology