Validation of the ICAR for the NIH MobileToolbox

The International Cognitive Ability Resource (ICAR) is the first well-validated nonproprietary cognitive assessment battery that freely distributes items to researchers (Condon & Revelle, 2014; Revelle et al., 2020) and has been used in dozens of research studies since its publication (Dworak et al., 2020). Four of the ICAR subtests, Progressive Matrices (PM), Three-Dimensional Rotation (R3D), Verbal Reasoning (VR) and Letter-and-Number Series (LNS), have been developed for smartphone administration within the Mobile Toolbox (MTB) platform alongside a library of many other cognitive measures (Gershon et al., 2022), The only convergent validity studies of the ICAR to date have been conducted using the 16-item Sample Test (ICAR16) (Condon & Revelle, 2014; Young & Keith, 2020), which includes four items from each of the four original item types (R3D; VR; LNS and a set of matrix reasoning items distinct from PM). The individual item sets of PM (64 items), R3D (66 items), VR (16 items), and LNS (9 items) have yet to be validated in their entirety, nor in their current MTB format. The format of the MTB ICAR subtests has been adapted in order to fit on a smartphone screen and improve test usability (e.g., inclusion of demo and practice items, touchscreen response selection). Furthermore, two of the subtests, PM and R3D, were developed as computer adaptive tests (CATs). These changes may significantly impact the way examinees interact with the items, and the psychometric properties cannot be assumed to be the same as those found in the original validation studies. Findings from initial convergent validity studies have also spurred additional questions about certain ICAR subtests. Young and Keith (2020) found that the four VR items from the ICAR16 correlated most strongly with tests of visual-spatial abilities on the WAIS-IV, which is inconsistent with theory given the items contain no visual elements. Conversely, the R3D items demonstrated lower than expected correlations with the WAIS-IV visual-spatial subtests. A validation of the MTB versions of the ICAR items against “gold standard” assessments is necessary for the tests to be useful for researchers. Furthermore, it will help clarify outstanding questions about the construct validity of the VR and R3D subtests.