Selection bias in college admissions test scores

Melissa Clark, Jesse Rothstein, Diane Whitmore Schanzenbach*

*Corresponding author for this work

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in "ACT states," where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.

Original languageEnglish (US)
Pages (from-to)295-307
Number of pages13
JournalEconomics of Education Review
Volume28
Issue number3
DOIs
StatePublished - Jun 1 2009

Fingerprint

trend
school
Test scores
Selection bias
College admissions
reform policy
Selectivity
student
participation

Keywords

  • College admissions test
  • Heckit
  • SAT
  • Selection bias

ASJC Scopus subject areas

  • Education
  • Economics and Econometrics

Cite this

Clark, Melissa ; Rothstein, Jesse ; Schanzenbach, Diane Whitmore. / Selection bias in college admissions test scores. In: Economics of Education Review. 2009 ; Vol. 28, No. 3. pp. 295-307.
@article{237cdb071b2b4e46aef9b8a1244365c0,
title = "Selection bias in college admissions test scores",
abstract = "Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in {"}ACT states,{"} where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.",
keywords = "College admissions test, Heckit, SAT, Selection bias",
author = "Melissa Clark and Jesse Rothstein and Schanzenbach, {Diane Whitmore}",
year = "2009",
month = "6",
day = "1",
doi = "10.1016/j.econedurev.2008.08.001",
language = "English (US)",
volume = "28",
pages = "295--307",
journal = "Economics of Education Review",
issn = "0272-7757",
publisher = "Elsevier Limited",
number = "3",

}

Selection bias in college admissions test scores. / Clark, Melissa; Rothstein, Jesse; Schanzenbach, Diane Whitmore.

In: Economics of Education Review, Vol. 28, No. 3, 01.06.2009, p. 295-307.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Selection bias in college admissions test scores

AU - Clark, Melissa

AU - Rothstein, Jesse

AU - Schanzenbach, Diane Whitmore

PY - 2009/6/1

Y1 - 2009/6/1

N2 - Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in "ACT states," where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.

AB - Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in "ACT states," where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.

KW - College admissions test

KW - Heckit

KW - SAT

KW - Selection bias

UR - http://www.scopus.com/inward/record.url?scp=67349277145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67349277145&partnerID=8YFLogxK

U2 - 10.1016/j.econedurev.2008.08.001

DO - 10.1016/j.econedurev.2008.08.001

M3 - Article

VL - 28

SP - 295

EP - 307

JO - Economics of Education Review

JF - Economics of Education Review

SN - 0272-7757

IS - 3

ER -