An Unbiased Correction for Sampling Error in Validity Generalization Studies

A correction for variance due to sampling error is part of all the statistical procedures used to study validity generalization. This correction is analogous to estimation of a variance component in a random effects analysis of variance. The correction usually has a large influence on the results of validity generalization analyses. James, Demaree, and Mulaik (1986) recently questioned the correction for sampling error, suggesting that several of the assumptions required for its derivation are questionable. In this article, I provide an alternative correction for sampling error that is shown to be (exactly) unbiased. A comparison of the calculations involved in the computation of the unbiased correction with the calculations involved in the calculation of the usual correction suggests that the 2 corrections will rarely differ substantially. Hence, the usual correction is not seriously biased.