Computerized questionnaires and the quality of survey data

Study Design. A retrospective data quality analysis was conducted. Objective. To compare missing response rates and internal consistency between computerized and paper surveys administered to spine patients. Summary of Background Data. Computerized patient surveys have been shown to offer numerous advantages over traditional paper surveys. It has been assumed that computerized surveys also improve data quality, but quantitative comparisons have not been made. Methods. Between January 1998 and December 2000, approximately 3500 computerized questionnaires and 15,000 paper questionnaires containing the MOS 36-Item Short-Form Health Survey (SF-36) and the Oswestry Low Back Pain Disability Questionnaire were administered in the National Spine Network. Missing response rates and the Response Consistency Index (RCI) were compared between computerized and paper questionnaire data. Results. Computer surveys had approximately half the missing response rate of paper surveys. For the SF-36, the computer survey had 1.7% missing, as compared with 3.3% missing on paper (P < 0.001). For the Oswestry, the computer survey had 2.9% missing, as compared with 6% missing on paper (P < 0.001). Whereas 84% of the SF-36 surveys and 85% of the Oswestry surveys collected by computer were completely filled out (no missing responses), only 68% of the SF-36 surveys (P < 0.001) and 77% of the Oswestry surveys (P < 0.001) collected on paper were completely filled out. The SF-36 data collected by computer had better internal consistency than the paper-form data, with average Response Consistency Index scores of 0.12 and 0.16, respectively (P = 0.001). Conclusions. Superior response rates and higher internal consistency suggest that computerized survey systems improve data quality, and may enhance instrument validity for commonly used measures of spine patient health.