Document Type


Publication Date




Emergency department (ED) return visits within 72 h may be a sign of poor quality of care and entail unnecessary use of healthcare resources. In this study, we compare the performance of two leading statistical and machine learning classification algorithms, and we use the best performing approach to identify novel risk factors of ED return visits.


We analyzed 3.2 million ED encounters with at least one diagnosis under “injury, poisoning and certain other consequences of external causes” and “external causes of morbidity.” These encounters included patients 18 years or older from across 128 emergency room facilities in the USA. For each encounter, we calculated the 72-h ED return status and retrieved 57 features from demographics, diagnoses, procedures, and medications administered during the process of administration of medical care. We implemented a mixed-effects model to assess the effects of the covariates while accounting for the hierarchical structure of the data. Additionally, we investigated the predictive accuracy of the extreme gradient boosting tree ensemble approach and compared the performance of the two methods.


The mixed-effects model indicates that certain blunt force and non-blunt trauma inflates the risk of a return visit. Notably, patients with trauma to the head and patients with burns and corrosions have elevated risks. This is in addition to 11 other classes of both blunt force and non-blunt force traumas. In addition, prior healthcare resource utilization, patients who have had one or more prior return visits within the last 6 months, prior ED visits, and the number of hospitalizations within the 6 months are associated with increased risk of returning to the ED after discharge. On the one hand, the area under the receiver characteristic curve (AUROC) of the mixed-effects model was 0.710 (0.707, 0.712). On the other hand, the gradient boosting tree ensemble had a lower AUROC of 0.698 CI (0.696, 0.700) on the independent test model.


The proposed mixed-effects model achieved the highest known AUC and resulted in the identification of novel risk factors. The model outperformed one of the leading machine learning ensemble classifiers, the extreme gradient boosting tree in terms of model performance. The risk factors we identified can assist emergency departments to decrease the number of unplanned return visits within 72 h.


This article was originally published in Journal of Orthopaedic Surgery and Research, volume 15, issue 1, in 2020.

Peer Reviewed



The authors

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.