Implementation of an Innovative Machine Learning-Based Triage Support Tool: Translating Technology and Research to Practice

Abstract

Session A presented Thursday, September 27, 10:00-11:00 am Purpose: Leveraging the electronic health record (EHR), as well as translating research to clinical practice, are paramount to improving the provision of healthcare in the US. The nurse-driven, often subjective, triage process can greatly benefit from improved use of technology. E-triage, a clinical decision support tool rooted in machine learning, has been shown to improve differentiation of patients with respect to clinical outcomes as compared to the Emergency Severity Index (ESI). The objective of the study is quantify and qualify our evolution from the traditional ESI triage system, to e-triage as an exemplar of successful transition from research to practice and nursing integration of data-driven clinical decision support. Design: We conducted a prospective cross-sectional study. Setting: The study took place at a level one urban, academic trauma center with approximately 70,000 patient visits annually. Participants/Subjects: Participants were all (65) practicing triage nurses with wide-ranging emergency department (ED) experience from 3 to over 30 years. The Institutional Review Board (IRB) approved this study. Methods: Over one year, we incrementally rolled out a new e-triage system. E-triage is a clinical decision support system that does not replace nursing assessment, but provides a triage level suggestion based on computed risk of several acute critical outcomes: mortality, intensive care unit (ICU) admission, and emergent procedure. Probabilistic risk is predicted from patients’ chief complaint, vital signs, and medical history and translated to a triage level recommendation. The machine learning algorithm is developed from the ED’s specific patient population adapting to practice environment factors such as resource availability, departmental objectives and risk tolerances. Statistical evaluation of e-triage included quantifying: (1) designed increases in the volume low-acuity fast-track patients, (2) increases in detection of critical acute outcomes as high-acuity, (3) patterns of agreement between e-triage recommendation and nurse assigned acuity. Results/Outcomes: Patient volume assigned to low-acuity (Level 4 and 5) increased by 55% (16% to 25%) through 12 months post-implementation indicating wide-spread adoption of e-triage amongst nursing. High-acuity (Level 1 and 2) detection of patients with critical acute outcomes increased by 16% (13% to 16%). In the high volume ED studied this amounts to an additional 225 patients per year receiving expedited up-front treatment (detected as high acuity) that will experience a critical outcome. Overall, total agreement between nursing and the e-triage system was 80%, 66% for high acuity, 83% for mid-acuity (Level 3), and 83% for low-acuity patients. Nurses tended to have higher agreement (>80%) with the e-triage acuity of common chief complaints such as “headache,” “chest pain” and “abdominal pain” and lower agreement for presentations such as “dizziness” or “seizures." Implications: Large scale practice change is possible, yet requires high levels of end-user engagement, including a mutually-educational relationship between the users and support technology. Trends in over-rides related to chief complaint and subsequent outcomes can continue to inform the implementation and development of the triage tool. This transition exemplifies the translation of informatics research to practice as well as the potential of nurses to use the EHR and machine learning to improve healthcare.