Comparing the Performance of Five Multidimensional CAT Selection Procedures With Different Stopping Rules

Lihua Yao*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

40 Scopus citations


Through simulated data, five multidimensional computerized adaptive testing (MCAT) selection procedures with varying test lengths are examined and compared using different stopping rules. Fixed item exposure rates are used for all the items, and the Priority Index (PI) method is used for the content constraints. Two stopping rules, standard error (SE) and predicted standard error reduction (PSER), are proposed; each MCAT selection process is stopped if either the required precision has been achieved or the selected number of items has reached the maximum limit. The five procedures are as follows: minimum angle (Ag), volume (Vm), minimize the error variance of the linear combination (V1), minimize the error variance of the composite score with the optimized weight (V2), and Kullback-Leibler (KL) information. The recovery for the domain scores or content scores and their overall score, test length, and test reliability are compared across the five MCAT procedures and between the two stopping rules. It is found that the two stopping rules are implemented successfully and that KL uses the least number of items to reach the same precision level, followed by Vm; Ag uses the largest number of items. On average, to reach a precision of SE =.35, 40, 55, 63, 63, and 82 items are needed for KL, Vm, V1, V2, and Ag, respectively, for the SE stopping rule. PSER yields 38, 45, 53, 58, and 68 items for KL, Vm, V1, V2, and Ag, respectively; PSER yields only slightly worse results than SE, but with much fewer items. Overall, KL is recommended for varying-length MCAT.

Original languageEnglish (US)
Pages (from-to)3-23
Number of pages21
JournalApplied Psychological Measurement
Issue number1
StatePublished - Jan 2013


  • CAT
  • domain scores
  • MCAT
  • MIRT
  • multidimensional information
  • multidimensional item response theory
  • Multidimensional Priority Index
  • overall scores

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)


Dive into the research topics of 'Comparing the Performance of Five Multidimensional CAT Selection Procedures With Different Stopping Rules'. Together they form a unique fingerprint.

Cite this