Pretest measures of the study outcome and the elimination of selection bias: Evidence from three within study comparisons

Kelly Hallberg*, Thomas D. Cook, Peter M. Steiner, M. H. Clark

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


This paper examines how pretest measures of a study outcome reduce selection bias in observational studies in education. The theoretical rationale for privileging pretests in bias control is that they are often highly correlated with the outcome, and in many contexts, they are also highly correlated with the selection process. To examine the pretest’s role in bias reduction, we use the data from two within study comparisons and an especially strong quasi-experiment, each with an educational intervention that seeks to improve achievement. In each study, the pretest measures are consistently highly correlated with post-intervention measures of themselves, but the studies vary the correlation between the pretest and the process of selection into treatment. Across the three datasets with two outcomes each, there are three cases where this correlation is low and three where it is high. A single wave of pretest always reduces bias across the six instances examined, and it eliminates bias in three of them. Adding a second pretest wave eliminates bias in two more instances. However, the pattern of bias elimination does not follow the predicted pattern—that more bias reduction ensues as a function of how highly the pretest is correlated with selection. The findings show that bias is more complexly related to the pretest’s correlation with selection than we hypothesized, and we seek to explain why.

Original languageEnglish (US)
Pages (from-to)274-283
Number of pages10
JournalPrevention Science
Issue number3
StatePublished - Apr 2018


  • Causal inference
  • Propensity score matching
  • Randomized experiment
  • Within-study comparison

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health


Dive into the research topics of 'Pretest measures of the study outcome and the elimination of selection bias: Evidence from three within study comparisons'. Together they form a unique fingerprint.

Cite this