Reliability from α to ω: A tutorial

William Revelle*, David M. Condon

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test-retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An online supplemental materials is provided with more detail and elaboration. The online supplemental materials is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability.

Original languageEnglish (US)
Pages (from-to)1395-1411
Number of pages17
JournalPsychological assessment
Volume31
Issue number12
DOIs
StatePublished - Dec 2019

Keywords

  • Classical test theory
  • Generalizability
  • R packages
  • Reliability

ASJC Scopus subject areas

  • Clinical Psychology
  • Psychiatry and Mental health

Fingerprint Dive into the research topics of 'Reliability from α to ω: A tutorial'. Together they form a unique fingerprint.

Cite this