AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics

Nan E. Rothrock; Michael Bass; Andrea Blumenthal; Richard C. Gershon; Beate Hanson; Alexander Joeris; Aaron Kaat; Suzanne Morrison; Robert V. O'Toole; Shalini Patel; Michael Stover; Michael J. Weaver; Raymond White; Maria Varela Diaz; Mark S. Vrahas

doi:10.2196/10880

AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics

Nan E. Rothrock^*, Michael Bass, Andrea Blumenthal, Richard C. Gershon, Beate Hanson, Alexander Joeris, Aaron Kaat, Suzanne Morrison, Robert V. O'Toole, Shalini Patel, Michael Stover, Michael J. Weaver, Raymond White, Maria Varela Diaz, Mark S. Vrahas

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

12 Scopus citations

Abstract

Background: Patient-reported outcomes are increasingly utilized in routine orthopedic clinical care. Computer adaptive tests (CATs) from the Patient-Reported Outcomes Measurement Information System (PROMIS) offer a brief and precise assessment that is well suited for collection within busy clinical environments. However, software apps that support the administration and scoring of CATs, provide immediate access to patient-reported outcome (PRO) scores, and minimize clinician burden are not widely available. Objective: Our objective was to design, implement, and test the feasibility and usability of a Web-based system for collecting CATs in orthopedic clinics. Methods: AO Patient Outcomes Center (AOPOC) was subjected to 2 rounds of testing. Alpha testing was conducted in 3 orthopedic clinics to evaluate ease of use and feasibility of integration in clinics. Patients completed an assessment of PROMIS CATs and a usability survey. Clinicians participated in a brief semistructured interview. Beta-phase testing evaluated system performance through load testing and usability of the updated version of AOPOC. In both rounds of testing, user satisfaction, bugs, change requests, and performance of PROMIS CATs were captured. Results: Patient feedback supported the ease of use in completing an assessment in AOPOC. Across both phases of testing, clinicians rated AOPOC as easy to use but noted difficulties in integrating a Web-based software application within their clinics. PROMIS CATs performed well; the default assessment of 2 CATs was completed quickly (mean 9.5 items) with a satisfactory range of measurement. Conclusion: AOPOC was demonstrated to be an easy-to-learn and easy-to-use software application for patients and clinicians that can be integrated into orthopedic clinical care. The workflow disruption in integrating any type of PRO collection must be addressed if patients' voices are to be better integrated in clinical care.

Original language	English (US)
Article number	e10880
Journal	JMIR Formative Research
Volume	3
Issue number	2
DOIs	https://doi.org/10.2196/10880
State	Published - Apr 2019

Funding

Financial support for this project was provided by AO North America and AO Trauma. AO Clinical Investigation and Documentation staff contributed to software design (BH) and manuscript preparation (AB, AJ). NER, MB, RCG, AK, SP, and MVD also receive grant funding for PROMIS from the National Institutes of Health (U2C CA186878).

Keywords

Orthopedics
Patient reported outcome measures
Tablet computers

ASJC Scopus subject areas

Health Informatics
Medicine (miscellaneous)

Access to Document

10.2196/10880

Cite this

Rothrock, N. E., Bass, M., Blumenthal, A., Gershon, R. C., Hanson, B., Joeris, A., Kaat, A., Morrison, S., O'Toole, R. V., Patel, S., Stover, M., Weaver, M. J., White, R., Diaz, M. V., & Vrahas, M. S. (2019). AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics. JMIR Formative Research, 3(2), Article e10880. https://doi.org/10.2196/10880

@article{612531f050664ab983b9d3f5ee815fb9,

title = "AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics",

abstract = "Background: Patient-reported outcomes are increasingly utilized in routine orthopedic clinical care. Computer adaptive tests (CATs) from the Patient-Reported Outcomes Measurement Information System (PROMIS) offer a brief and precise assessment that is well suited for collection within busy clinical environments. However, software apps that support the administration and scoring of CATs, provide immediate access to patient-reported outcome (PRO) scores, and minimize clinician burden are not widely available. Objective: Our objective was to design, implement, and test the feasibility and usability of a Web-based system for collecting CATs in orthopedic clinics. Methods: AO Patient Outcomes Center (AOPOC) was subjected to 2 rounds of testing. Alpha testing was conducted in 3 orthopedic clinics to evaluate ease of use and feasibility of integration in clinics. Patients completed an assessment of PROMIS CATs and a usability survey. Clinicians participated in a brief semistructured interview. Beta-phase testing evaluated system performance through load testing and usability of the updated version of AOPOC. In both rounds of testing, user satisfaction, bugs, change requests, and performance of PROMIS CATs were captured. Results: Patient feedback supported the ease of use in completing an assessment in AOPOC. Across both phases of testing, clinicians rated AOPOC as easy to use but noted difficulties in integrating a Web-based software application within their clinics. PROMIS CATs performed well; the default assessment of 2 CATs was completed quickly (mean 9.5 items) with a satisfactory range of measurement. Conclusion: AOPOC was demonstrated to be an easy-to-learn and easy-to-use software application for patients and clinicians that can be integrated into orthopedic clinical care. The workflow disruption in integrating any type of PRO collection must be addressed if patients' voices are to be better integrated in clinical care.",

keywords = "Orthopedics, Patient reported outcome measures, Tablet computers",

author = "Rothrock, {Nan E.} and Michael Bass and Andrea Blumenthal and Gershon, {Richard C.} and Beate Hanson and Alexander Joeris and Aaron Kaat and Suzanne Morrison and O'Toole, {Robert V.} and Shalini Patel and Michael Stover and Weaver, {Michael J.} and Raymond White and Diaz, {Maria Varela} and Vrahas, {Mark S.}",

year = "2019",

month = apr,

doi = "10.2196/10880",

language = "English (US)",

volume = "3",

journal = "JMIR Formative Research",

issn = "2561-326X",

publisher = "JMIR Publications Inc.",

number = "2",

}

Rothrock, NE , Bass, M, Blumenthal, A, Gershon, RC, Hanson, B, Joeris, A, Kaat, A, Morrison, S, O'Toole, RV, Patel, S, Stover, M, Weaver, MJ, White, R, Diaz, MV & Vrahas, MS 2019, 'AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics', JMIR Formative Research, vol. 3, no. 2, e10880. https://doi.org/10.2196/10880

AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics. / Rothrock, Nan E.; Bass, Michael; Blumenthal, Andrea et al.
In: JMIR Formative Research, Vol. 3, No. 2, e10880, 04.2019.

Research output: Contribution to journal › Article › peer-review

TY - JOUR

T1 - AO Patient outcomes center

T2 - Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics

AU - Rothrock, Nan E.

AU - Bass, Michael

AU - Blumenthal, Andrea

AU - Gershon, Richard C.

AU - Hanson, Beate

AU - Joeris, Alexander

AU - Kaat, Aaron

AU - Morrison, Suzanne

AU - O'Toole, Robert V.

AU - Patel, Shalini

AU - Stover, Michael

AU - Weaver, Michael J.

AU - White, Raymond

AU - Diaz, Maria Varela

AU - Vrahas, Mark S.

PY - 2019/4

Y1 - 2019/4

N2 - Background: Patient-reported outcomes are increasingly utilized in routine orthopedic clinical care. Computer adaptive tests (CATs) from the Patient-Reported Outcomes Measurement Information System (PROMIS) offer a brief and precise assessment that is well suited for collection within busy clinical environments. However, software apps that support the administration and scoring of CATs, provide immediate access to patient-reported outcome (PRO) scores, and minimize clinician burden are not widely available. Objective: Our objective was to design, implement, and test the feasibility and usability of a Web-based system for collecting CATs in orthopedic clinics. Methods: AO Patient Outcomes Center (AOPOC) was subjected to 2 rounds of testing. Alpha testing was conducted in 3 orthopedic clinics to evaluate ease of use and feasibility of integration in clinics. Patients completed an assessment of PROMIS CATs and a usability survey. Clinicians participated in a brief semistructured interview. Beta-phase testing evaluated system performance through load testing and usability of the updated version of AOPOC. In both rounds of testing, user satisfaction, bugs, change requests, and performance of PROMIS CATs were captured. Results: Patient feedback supported the ease of use in completing an assessment in AOPOC. Across both phases of testing, clinicians rated AOPOC as easy to use but noted difficulties in integrating a Web-based software application within their clinics. PROMIS CATs performed well; the default assessment of 2 CATs was completed quickly (mean 9.5 items) with a satisfactory range of measurement. Conclusion: AOPOC was demonstrated to be an easy-to-learn and easy-to-use software application for patients and clinicians that can be integrated into orthopedic clinical care. The workflow disruption in integrating any type of PRO collection must be addressed if patients' voices are to be better integrated in clinical care.

AB - Background: Patient-reported outcomes are increasingly utilized in routine orthopedic clinical care. Computer adaptive tests (CATs) from the Patient-Reported Outcomes Measurement Information System (PROMIS) offer a brief and precise assessment that is well suited for collection within busy clinical environments. However, software apps that support the administration and scoring of CATs, provide immediate access to patient-reported outcome (PRO) scores, and minimize clinician burden are not widely available. Objective: Our objective was to design, implement, and test the feasibility and usability of a Web-based system for collecting CATs in orthopedic clinics. Methods: AO Patient Outcomes Center (AOPOC) was subjected to 2 rounds of testing. Alpha testing was conducted in 3 orthopedic clinics to evaluate ease of use and feasibility of integration in clinics. Patients completed an assessment of PROMIS CATs and a usability survey. Clinicians participated in a brief semistructured interview. Beta-phase testing evaluated system performance through load testing and usability of the updated version of AOPOC. In both rounds of testing, user satisfaction, bugs, change requests, and performance of PROMIS CATs were captured. Results: Patient feedback supported the ease of use in completing an assessment in AOPOC. Across both phases of testing, clinicians rated AOPOC as easy to use but noted difficulties in integrating a Web-based software application within their clinics. PROMIS CATs performed well; the default assessment of 2 CATs was completed quickly (mean 9.5 items) with a satisfactory range of measurement. Conclusion: AOPOC was demonstrated to be an easy-to-learn and easy-to-use software application for patients and clinicians that can be integrated into orthopedic clinical care. The workflow disruption in integrating any type of PRO collection must be addressed if patients' voices are to be better integrated in clinical care.

KW - Orthopedics

KW - Patient reported outcome measures

KW - Tablet computers

UR - http://www.scopus.com/inward/record.url?scp=85096866491&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85096866491&partnerID=8YFLogxK

U2 - 10.2196/10880

DO - 10.2196/10880

M3 - Article

C2 - 30977735

AN - SCOPUS:85096866491

SN - 2561-326X

VL - 3

JO - JMIR Formative Research

JF - JMIR Formative Research

IS - 2

M1 - e10880

ER -

AO Patient outcomes center: Design, implementation, and evaluation of a software application for the collection of patient-reported outcome measures in orthopedic outpatient clinics

Abstract

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this