Evaluation of a hospital admission prediction model adding coded chief complaint data using neural network methodology

Objective Our objective was to apply neural network methodology to determine whether adding coded chief complaint (CCC) data to triage information would result in an improved hospital admission prediction model than one without CCC data. Participants and methods We carried out a retrospective derivation and validation cohort study of all adult emergency department visits to a single center. We downloaded triage, chief complaint, and admission/ discharge data on each included visit. Using a CCC algorithm and the Levenberg-Marquardt back-propagation learning method, we derived hospital admission prediction models without and with CCC data and applied these to the validation cohort, reporting the prediction models' characteristics. Results A total of 74 056 emergency department visits were included in the derivation cohort, 85 144 in the validation cohort with 213 CCC categories. The sensitivity/ specificity of the derivation cohort models without and with CCC data were 64.0% [95% confidence interval (CI): 63.7-64.3], 87.7% (95% CI: 87.4-88.0), 59.8% (95% CI: 59.5-60.3%), and 91.7% (95% CI: 91.4-92.0) respectively. The sensitivity/specificity of the derived models without and with CCC data applied to the validation cohort were 60.7% (95% CI: 60.4-61.0), 87.7% (95% CI: 87.4-88.0), 59.8% (95% CI: 59.5-60.3), and 90.6% (95% CI: 90.3-90.9) respectively. The area under the curve in the validation cohort for the derived models without and with CCC data were 0.840 (95% CI: 0.838-0.842) and 0.860 (95% CI: 0.858-0.862). Net reclassification index (0.156; 95% CI: 0.148-0.163) and integrated discrimination improvement (0.060; 95% CI: 0.058-0.061) in the CCC model were significant. Conclusion Neural net methodology application resulted in the derivation and validation of a modestly stronger hospital admission prediction model after the addition of CCC data.