Does Scoring Method Impact Estimation of Significant Individual Changes Assessed by Patient-Reported Outcome Measures? Comparing Classical Test Theory Versus Item Response Theory

Xiaodan Tang*, Benjamin David Schalet, John Devin Peipert, David Cella

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Objectives: This study aimed to examine the ability of classical test theory (CTT) and item response theory (IRT) scores assessed by Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures to identify significant individual changes in the setting of clinical studies, using both simulated and empirical data. Methods: We used simulated data to compare the estimation of significant individual changes between CTT and IRT scores across different conditions and a clinical trial data set to verify the simulation results. We calculated reliable change indexes to estimate significant individual changes. Results: For small true change, IRT scores showed a slightly higher rate of classifying change groups than CTT scores and were comparable with CTT scores for a shorter test length. Additionally, IRT scores were found to have a prominent advantage in the classification rates of change groups for medium to high true change over CTT scores. Such an advantage became prominent in a longer test length. The empirical data analysis results using an anchor-based approach further supported the above findings that IRT scores can more accurately classify participants into change groups than CTT scores. Conclusions: Given that IRT scores perform better, or at least comparably, in most conditions, we recommend using IRT scores to estimate significant individual changes and identify responders to treatment. This study provides evidence-based guidance in detecting individual changes based on CTT and IRT scores under various measurement conditions and leads to recommendations for identifying responders to treatment for participants in clinical trials.

Original languageEnglish (US)
Pages (from-to)1518-1524
Number of pages7
JournalValue in Health
Volume26
Issue number10
DOIs
StatePublished - Oct 2023

Keywords

  • classical test theory
  • item response theory
  • patient-reported outcomes
  • reliable change index
  • significant individual changes

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health
  • Health Policy

Fingerprint

Dive into the research topics of 'Does Scoring Method Impact Estimation of Significant Individual Changes Assessed by Patient-Reported Outcome Measures? Comparing Classical Test Theory Versus Item Response Theory'. Together they form a unique fingerprint.

Cite this