How large an effect can we expect from school reforms?

Spyros Konstantopoulos*, Larry Vernon Hedges

*Corresponding author for this work

Research output: Contribution to journalArticle

30 Citations (Scopus)

Abstract

Background/Context: Determining the effectiveness of reform strategies is a major part of the current and future educational research agenda. Effects of education reforms will be evaluated largely quantitatively, and an important aspect of this work will be judging how well reform strategies work. The rhetoric of contemporary school reform suggests two somewhat different solutions to the problem of the interpretive frame. One solution is derived from the idea that the goal of school reform is to reduce the achievement gaps between minority groups such as Blacks or Hispanics and Whites, rich and poor, and males and females. The second solution is derived from a similar idea that school reforms are intended to reduce the achievement gap between lower and higher achieving schools. Purpose: The purpose of this paper is to explore these two alternative frameworks for interpreting the effects of school reforms and to gain insight about the implications of each of the frameworks for interpreting these effects. Research Design: We use NAEP trend data and examine empirical evidence about the implications of these two frameworks for judging the effects of school reforms. Our study is correlational and uses observational data from the 1970s, 1980s, and 1990s. Findings: We find that these two frameworks lead to different judgments about whether the effects of reforms are large enough to be important. We argue that the normative distribution of school effects framework is more appropriate than the other framework for interpret-ing the likely magnitude of school reform effects. In addition, we show that interpreting the magnitude of the effects of school reforms in terms of individual variation and the achievement gaps between important student groups may not only be disappointing, but also misleading. Conclusions/Recommendations: The results of this study can be used to provide a way to obtain plausible reform effects for designing studies of school reform. The results of this study can also provide a context in which to evaluate the results of studies of the effectiveness of school reform in terms of national data. Our study also illustrates one way in which survey data can contribute to evidence-based policy formation. The distribution of the observed school effects provides a basis for estimating plausible effect magnitudes far planning intervention studies. The analyses of school effects can also provide a context for interpreting treatment effects within the context of the observed variation. It permits the policy researcher to explain the implications of reform effects within the backdrop of observed variation within which any intervention will operate.

Original languageEnglish (US)
Pages (from-to)1611-1638
Number of pages28
JournalTeachers College Record
Volume110
Issue number8
StatePublished - Aug 1 2008

Fingerprint

school reform
reform strategy
reform
school
educational research
research planning
evidence
rhetoric

ASJC Scopus subject areas

  • Education

Cite this

Konstantopoulos, Spyros ; Hedges, Larry Vernon. / How large an effect can we expect from school reforms?. In: Teachers College Record. 2008 ; Vol. 110, No. 8. pp. 1611-1638.
@article{850baa5721b24f53af7b965bcc20d265,
title = "How large an effect can we expect from school reforms?",
abstract = "Background/Context: Determining the effectiveness of reform strategies is a major part of the current and future educational research agenda. Effects of education reforms will be evaluated largely quantitatively, and an important aspect of this work will be judging how well reform strategies work. The rhetoric of contemporary school reform suggests two somewhat different solutions to the problem of the interpretive frame. One solution is derived from the idea that the goal of school reform is to reduce the achievement gaps between minority groups such as Blacks or Hispanics and Whites, rich and poor, and males and females. The second solution is derived from a similar idea that school reforms are intended to reduce the achievement gap between lower and higher achieving schools. Purpose: The purpose of this paper is to explore these two alternative frameworks for interpreting the effects of school reforms and to gain insight about the implications of each of the frameworks for interpreting these effects. Research Design: We use NAEP trend data and examine empirical evidence about the implications of these two frameworks for judging the effects of school reforms. Our study is correlational and uses observational data from the 1970s, 1980s, and 1990s. Findings: We find that these two frameworks lead to different judgments about whether the effects of reforms are large enough to be important. We argue that the normative distribution of school effects framework is more appropriate than the other framework for interpret-ing the likely magnitude of school reform effects. In addition, we show that interpreting the magnitude of the effects of school reforms in terms of individual variation and the achievement gaps between important student groups may not only be disappointing, but also misleading. Conclusions/Recommendations: The results of this study can be used to provide a way to obtain plausible reform effects for designing studies of school reform. The results of this study can also provide a context in which to evaluate the results of studies of the effectiveness of school reform in terms of national data. Our study also illustrates one way in which survey data can contribute to evidence-based policy formation. The distribution of the observed school effects provides a basis for estimating plausible effect magnitudes far planning intervention studies. The analyses of school effects can also provide a context for interpreting treatment effects within the context of the observed variation. It permits the policy researcher to explain the implications of reform effects within the backdrop of observed variation within which any intervention will operate.",
author = "Spyros Konstantopoulos and Hedges, {Larry Vernon}",
year = "2008",
month = "8",
day = "1",
language = "English (US)",
volume = "110",
pages = "1611--1638",
journal = "Teachers College Record",
issn = "0161-4681",
publisher = "Teachers College Record",
number = "8",

}

Konstantopoulos, S & Hedges, LV 2008, 'How large an effect can we expect from school reforms?', Teachers College Record, vol. 110, no. 8, pp. 1611-1638.

How large an effect can we expect from school reforms? / Konstantopoulos, Spyros; Hedges, Larry Vernon.

In: Teachers College Record, Vol. 110, No. 8, 01.08.2008, p. 1611-1638.

Research output: Contribution to journalArticle

TY - JOUR

T1 - How large an effect can we expect from school reforms?

AU - Konstantopoulos, Spyros

AU - Hedges, Larry Vernon

PY - 2008/8/1

Y1 - 2008/8/1

N2 - Background/Context: Determining the effectiveness of reform strategies is a major part of the current and future educational research agenda. Effects of education reforms will be evaluated largely quantitatively, and an important aspect of this work will be judging how well reform strategies work. The rhetoric of contemporary school reform suggests two somewhat different solutions to the problem of the interpretive frame. One solution is derived from the idea that the goal of school reform is to reduce the achievement gaps between minority groups such as Blacks or Hispanics and Whites, rich and poor, and males and females. The second solution is derived from a similar idea that school reforms are intended to reduce the achievement gap between lower and higher achieving schools. Purpose: The purpose of this paper is to explore these two alternative frameworks for interpreting the effects of school reforms and to gain insight about the implications of each of the frameworks for interpreting these effects. Research Design: We use NAEP trend data and examine empirical evidence about the implications of these two frameworks for judging the effects of school reforms. Our study is correlational and uses observational data from the 1970s, 1980s, and 1990s. Findings: We find that these two frameworks lead to different judgments about whether the effects of reforms are large enough to be important. We argue that the normative distribution of school effects framework is more appropriate than the other framework for interpret-ing the likely magnitude of school reform effects. In addition, we show that interpreting the magnitude of the effects of school reforms in terms of individual variation and the achievement gaps between important student groups may not only be disappointing, but also misleading. Conclusions/Recommendations: The results of this study can be used to provide a way to obtain plausible reform effects for designing studies of school reform. The results of this study can also provide a context in which to evaluate the results of studies of the effectiveness of school reform in terms of national data. Our study also illustrates one way in which survey data can contribute to evidence-based policy formation. The distribution of the observed school effects provides a basis for estimating plausible effect magnitudes far planning intervention studies. The analyses of school effects can also provide a context for interpreting treatment effects within the context of the observed variation. It permits the policy researcher to explain the implications of reform effects within the backdrop of observed variation within which any intervention will operate.

AB - Background/Context: Determining the effectiveness of reform strategies is a major part of the current and future educational research agenda. Effects of education reforms will be evaluated largely quantitatively, and an important aspect of this work will be judging how well reform strategies work. The rhetoric of contemporary school reform suggests two somewhat different solutions to the problem of the interpretive frame. One solution is derived from the idea that the goal of school reform is to reduce the achievement gaps between minority groups such as Blacks or Hispanics and Whites, rich and poor, and males and females. The second solution is derived from a similar idea that school reforms are intended to reduce the achievement gap between lower and higher achieving schools. Purpose: The purpose of this paper is to explore these two alternative frameworks for interpreting the effects of school reforms and to gain insight about the implications of each of the frameworks for interpreting these effects. Research Design: We use NAEP trend data and examine empirical evidence about the implications of these two frameworks for judging the effects of school reforms. Our study is correlational and uses observational data from the 1970s, 1980s, and 1990s. Findings: We find that these two frameworks lead to different judgments about whether the effects of reforms are large enough to be important. We argue that the normative distribution of school effects framework is more appropriate than the other framework for interpret-ing the likely magnitude of school reform effects. In addition, we show that interpreting the magnitude of the effects of school reforms in terms of individual variation and the achievement gaps between important student groups may not only be disappointing, but also misleading. Conclusions/Recommendations: The results of this study can be used to provide a way to obtain plausible reform effects for designing studies of school reform. The results of this study can also provide a context in which to evaluate the results of studies of the effectiveness of school reform in terms of national data. Our study also illustrates one way in which survey data can contribute to evidence-based policy formation. The distribution of the observed school effects provides a basis for estimating plausible effect magnitudes far planning intervention studies. The analyses of school effects can also provide a context for interpreting treatment effects within the context of the observed variation. It permits the policy researcher to explain the implications of reform effects within the backdrop of observed variation within which any intervention will operate.

UR - http://www.scopus.com/inward/record.url?scp=51549101284&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=51549101284&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:51549101284

VL - 110

SP - 1611

EP - 1638

JO - Teachers College Record

JF - Teachers College Record

SN - 0161-4681

IS - 8

ER -