TY - JOUR
T1 - Eye-tracking retrospective think-aloud as a novel approach for a usability evaluation
AU - Cho, Hwayoung
AU - Powell, Dakota
AU - Pichon, Adrienne
AU - Kuhns, Lisa M.
AU - Garofalo, Robert
AU - Schnall, Rebecca
N1 - Funding Information:
This research was supported by the National Institute of Minority and Health Disparities of the National Institutes of Health (NIH) under award number U01MD11279 (MPI: RS and RG). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Publisher Copyright:
© 2019 Elsevier B.V.
PY - 2019/9
Y1 - 2019/9
N2 - Objective: To report on the use of an eye-tracking retrospective think-aloud for usability evaluation and to describe its application in assessing the usability of a mobile health app. Materials and Methods: We used an eye-tracking retrospective think-aloud to evaluate the usability of an HIV prevention mobile app among 20 young men (15–18 years) in New York City, NY; Birmingham, AL; and Chicago, IL. Task performance metrics, critical errors, a task completion rate per participant, and a task completion rate per task, were measured. Eye-tracking metrics including fixation, saccades, time to first fixation, time spent, and revisits were measured and compared among participants with/without a critical error. Results: Using task performance analysis, we identified 19 critical errors on four activities, and of those, two activities had a task completion rate of less than 78%. To better understand these usability issues, we thoroughly analyzed participants’ corresponding eye movements and verbal comments using an in-depth problem analysis. In areas of interest created for the activity with critical usability problems, there were significant differences in time spent (p = 0.008), revisits (p = 0.004), and total numbers of fixations (p = 0.007) by participants with/without a critical error. The overall mean score of perceived usability rated by the Health IT Usability Evaluation Scale was 4.64 (SD = 0.33), reflecting strong usability of the app. Discussion and Conclusion: An eye-tracking retrospective think-aloud enabled us to identify critical usability problems as well as gain an in-depth understanding of the usability issues related to interactions between end-users and the app. Findings from this study highlight the utility of an eye-tracking retrospective think-aloud in consumer health usability evaluation research.
AB - Objective: To report on the use of an eye-tracking retrospective think-aloud for usability evaluation and to describe its application in assessing the usability of a mobile health app. Materials and Methods: We used an eye-tracking retrospective think-aloud to evaluate the usability of an HIV prevention mobile app among 20 young men (15–18 years) in New York City, NY; Birmingham, AL; and Chicago, IL. Task performance metrics, critical errors, a task completion rate per participant, and a task completion rate per task, were measured. Eye-tracking metrics including fixation, saccades, time to first fixation, time spent, and revisits were measured and compared among participants with/without a critical error. Results: Using task performance analysis, we identified 19 critical errors on four activities, and of those, two activities had a task completion rate of less than 78%. To better understand these usability issues, we thoroughly analyzed participants’ corresponding eye movements and verbal comments using an in-depth problem analysis. In areas of interest created for the activity with critical usability problems, there were significant differences in time spent (p = 0.008), revisits (p = 0.004), and total numbers of fixations (p = 0.007) by participants with/without a critical error. The overall mean score of perceived usability rated by the Health IT Usability Evaluation Scale was 4.64 (SD = 0.33), reflecting strong usability of the app. Discussion and Conclusion: An eye-tracking retrospective think-aloud enabled us to identify critical usability problems as well as gain an in-depth understanding of the usability issues related to interactions between end-users and the app. Findings from this study highlight the utility of an eye-tracking retrospective think-aloud in consumer health usability evaluation research.
KW - Eye movement measurements
KW - Eye movements
KW - Eye-tracking
KW - Health IT
KW - Information technology
KW - Mobile applications
KW - Mobile health
KW - Usability evaluation
UR - http://www.scopus.com/inward/record.url?scp=85069713240&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85069713240&partnerID=8YFLogxK
U2 - 10.1016/j.ijmedinf.2019.07.010
DO - 10.1016/j.ijmedinf.2019.07.010
M3 - Article
C2 - 31445278
AN - SCOPUS:85069713240
SN - 1386-5056
VL - 129
SP - 366
EP - 373
JO - International Journal of Medical Informatics
JF - International Journal of Medical Informatics
ER -