Test-retest reliability of the Newest Vital Sign health literacy instrument: In-person and remote administration

Objective: To determine the reliability of the Newest Vital Sign (NVS) administered via telephone by examining test-retest properties of the measure. Methods: Data were obtained from a randomized controlled trial promoting opioid safe use. Participants were 18 or older and English-speaking. NVS assessment occurred in-person at baseline and in-person or via telephone at follow-up. Intraclass correlation coefficients (ICCs) were used to assess the test-retest reliability using raw NVS scores by mode of administration of the second NVS assessment. Kappa statistics were used to examine test-retest agreement based on categorized NVS score. Internal consistency was measured with Cronbach's alpha. Results: Data from 216 patients (70 completing follow-up in-person and 146 via telephone) were included. Reliability was high (ICCs: in-person = 0.81, phone = 0.70). Agreement was lower for three category NVS score (Kappas: in-person = 0.58, 95% CI [0.39-0.77]; phone = 0.52, 95% CI [0.39-0.65]) compared to two category NVS (Kappas: in-person = 0.65, 95% CI [0.46-0.85]; phone = 0.64, 95% CI [0.51-0.78]). Correlations decreased as time between administrations increased. Internal consistency was moderately high (baseline NVS in-person (α = 0.76), follow-up NVS in-person (α = 0.76), and phone follow-up (α = 0.78). Conclusion: The test-retest properties of the NVS are similar by mode of administration. Practice implications: This data suggests the NVS measure is reliably administered by telephone.