A Review of Statistical Methods for Generalizing From Evaluations of Educational Interventions

Elizabeth Tipton, Robert B. Olsen

Research output: Contribution to journalReview article

1 Citation (Scopus)

Abstract

School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected randomly from a target population. This paper provides an overview of statistical methods for improving generalizations from intervention research in education. These are presented as a series of steps aimed at improving research design—particularly recruitment—as well as methods for assessing and summarizing generalizability and estimating treatment impacts for clearly defined target populations.

Original languageEnglish (US)
Pages (from-to)516-524
Number of pages9
JournalEducational Researcher
Volume47
Issue number8
DOIs
StatePublished - Nov 1 2018

Fingerprint

statistical method
evaluation
school
education
evidence
student

Keywords

  • educational policy
  • evaluation
  • experimental design
  • experimental research
  • external validity
  • generalizability
  • multisite studies
  • policy
  • program evaluation
  • propensity scores
  • research methodology
  • sampling
  • statistics

ASJC Scopus subject areas

  • Education

Cite this

@article{a93d2d01e7034d2ab2071bb580083602,
title = "A Review of Statistical Methods for Generalizing From Evaluations of Educational Interventions",
abstract = "School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected randomly from a target population. This paper provides an overview of statistical methods for improving generalizations from intervention research in education. These are presented as a series of steps aimed at improving research design—particularly recruitment—as well as methods for assessing and summarizing generalizability and estimating treatment impacts for clearly defined target populations.",
keywords = "educational policy, evaluation, experimental design, experimental research, external validity, generalizability, multisite studies, policy, program evaluation, propensity scores, research methodology, sampling, statistics",
author = "Elizabeth Tipton and Olsen, {Robert B.}",
year = "2018",
month = "11",
day = "1",
doi = "10.3102/0013189X18781522",
language = "English (US)",
volume = "47",
pages = "516--524",
journal = "Educational Researcher",
issn = "0013-189X",
publisher = "SAGE Publications Inc.",
number = "8",

}

A Review of Statistical Methods for Generalizing From Evaluations of Educational Interventions. / Tipton, Elizabeth; Olsen, Robert B.

In: Educational Researcher, Vol. 47, No. 8, 01.11.2018, p. 516-524.

Research output: Contribution to journalReview article

TY - JOUR

T1 - A Review of Statistical Methods for Generalizing From Evaluations of Educational Interventions

AU - Tipton, Elizabeth

AU - Olsen, Robert B.

PY - 2018/11/1

Y1 - 2018/11/1

N2 - School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected randomly from a target population. This paper provides an overview of statistical methods for improving generalizations from intervention research in education. These are presented as a series of steps aimed at improving research design—particularly recruitment—as well as methods for assessing and summarizing generalizability and estimating treatment impacts for clearly defined target populations.

AB - School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected randomly from a target population. This paper provides an overview of statistical methods for improving generalizations from intervention research in education. These are presented as a series of steps aimed at improving research design—particularly recruitment—as well as methods for assessing and summarizing generalizability and estimating treatment impacts for clearly defined target populations.

KW - educational policy

KW - evaluation

KW - experimental design

KW - experimental research

KW - external validity

KW - generalizability

KW - multisite studies

KW - policy

KW - program evaluation

KW - propensity scores

KW - research methodology

KW - sampling

KW - statistics

UR - http://www.scopus.com/inward/record.url?scp=85049025729&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049025729&partnerID=8YFLogxK

U2 - 10.3102/0013189X18781522

DO - 10.3102/0013189X18781522

M3 - Review article

VL - 47

SP - 516

EP - 524

JO - Educational Researcher

JF - Educational Researcher

SN - 0013-189X

IS - 8

ER -