Abstract
Artificial intelligence plays a central role in patient triage by enhancing the accuracy and efficiency of ranking care, allowing rapid identification of critically ill patients, reducing under- and over-triaging, and enhancing resource distribution in clinical settings, which eventually improves patient outcomes and reduces delay times. This study aimed to assess and summarize the current evidence on how artificial intelligence (AI), particularly machine learning (ML) models, are used to improve the accuracy of triage and predict patient outcomes in Emergency Departments (EDs). A widespread search was conducted across three major scientific databases, targeting studies published between 2023 and 2024. The search strategy combined keywords related to AI, ML, ED, triage, and patient outcomes. The studies evaluated a broad range of patient variables, including demographic characteristics (age, gender, ethnicity, socioeconomic status), vital signs (heart rate, respiratory rate, blood pressure, oxygen saturation, body temperature), medical history, symptoms, laboratory results, imaging data (CT scans, ECGs, slit lamp images), and emergency visit details. ML and AI models generally enhanced triage accuracy, with some achieving high performance metrics (e.g., 91% AUC and 70% F1 score using Histogram-Based Gradient Boosting classifiers) and effectively predicting critical outcomes, such as intubation need, ICU admission, in-hospital cardiac arrest, and vasopressor administration. ChatGPT showed promise in specialized triage contexts, such as metastatic prostate cancer; however, it had notable under-triage rates in high-acuity groups. AI-assisted imaging significantly improved sensitivity in detecting conditions, such as Inferior Vena Cava Embolism, without loss of specificity. In emergency eye care, AI combined with ocular imaging was beneficial but limited to that specialty. Overall, AI and ML models demonstrated positive impacts on triage efficacy and patient outcome prediction across diverse emergency care settings. These improvements translate into better identification of critically ill patients and more efficient use of ED resources.
Key words: Artificial Intelligence, Clinical Decision-Making, Emergency Service, Hospital, Machine Learning, Triage
Introduction
Triage, derived from the French word "trier," meaning to sort or organize, is a process used in healthcare to prioritize patients based on the severity of their conditions, determining the order in which they should receive care and monitoring (1). Triage is a systematic process that sorts patients based on the severity of their condition to ensure that those who need urgent care receive it promptly (1). It relies on rapid assessment, standardized classification systems, and ongoing monitoring to manage patient flow efficiently in busy healthcare environments (2). Artificial intelligence (AI) has significantly transformed traditional triage methods in emergency settings by enhancing accuracy, efficiency, and consistency in patient assessment (3).
Traditional triage heavily relies on subjective clinician decisions that can vary with clinician knowledge and workload, while AI-driven triage offers standardized, data-driven evaluations without exhaustion. This reduces inconsistency and human error (1, 2). While AI offers promising enhancements to triage, some disadvantages highlight the need for cautious use, such as Over-triage and Under-triage Risks, Technology and Automation Bias, as well as Ethical and Equity Concerns (4).
AI encompasses various approaches and techniques designed to enable machines to perform tasks that typically require human intelligence (5). The key types of AI include: Machine Learning (ML), Deep Learning (DL), Expert Systems, Natural Language Processing (NLP), and Computer Vision (5). However, challenges remain, including ethical concerns about data sharing and regulatory issues, such as potential risks from overconfident AI algorithms, which could lead to adverse patient outcomes. There is a need for proper education on AI's limitations and integration into healthcare systems to avoid errors and ensure quality improvement (6). This study aimed to assess and summarize the current evidence on how AI, particularly ML models, are used to improve the accuracy of triage and predict patient outcomes in emergency departments (EDs).
Methods
This narrative review was conducted using a widespread search across three major scientific databases, namely PubMed, Scopus, and Web of Science, targeting studies published between 2023 and 2024. The search strategy combined keywords related to artificial intelligence (AI), machine learning (ML), emergency departments (EDs), triage, and patient outcomes. Inclusion criteria comprised original research articles explicitly addressing AI or ML applications in emergency care triage and patient outcome prediction. On the other hand, review articles, studies without outcome data, non-English publications, and outdated studies were excluded from the study. The selected articles were then screened and analyzed to extract relevant data on the types of AI/ML techniques used, their impact on triage accuracy, and patient outcome improvements.
The study protocol was approved by the Ethics Committee under the code IR.SSU.MEDICINE.REC.1403.289.
Results
The studies on the role of AI in triage are presented in Table 1.
This review analyzed 19 studies, including retrospective cohort and prospective designs. In total, 14 of these focused on ML models applied to ED triage and patient outcome prediction, while the remainder examined AI systems not solely based on ML, such as rule-based AI triage and chatbot-based AI (e.g., ChatGPT). The studies evaluated a broad range of patient variables, such as demographic characteristics (age, gender, ethnicity, socioeconomic status), vital signs (heart rate, respiratory rate, blood pressure, oxygen saturation, body temperature), medical history, symptoms, laboratory results, imaging data (CT scans, ECGs, slit lamp images), and emergency visit details. ML and AI models generally enhanced triage accuracy, with some achieving high performance metrics (e.g., 91% AUC and 70% F1 score using Histogram-Based Gradient Boosting Classifiers) and effectively predicting critical outcomes, such as intubation need, ICU admission, in-hospital cardiac arrest, and vasopressor administration.
ChatGPT showed promise in specialized triage contexts, including metastatic prostate cancer; however, it had notable under-triage rates in high-acuity groups. AI-assisted imaging significantly improved sensitivity in detecting conditions, such as Inferior Vena Cava Embolism, without loss of specificity. In emergency eye care, AI combined with ocular imaging was beneficial but limited to that specialty. The application of machine learning models for mortality prediction is presented in Table 2.
Table 1. The studies on the role of AI in triage accuracy
Author |
Type of AI/ML were used |
Variables measured |
Was it beneficial in improving the triage efficacy? |
Brandao-de-Resende, 2023
(7) |
DemDx Ophthalmology Triage System (DOTS)
(supervised ML) |
Age and possibly socioeconomic status, Medical data |
Similar sensitivity to triage nurses, with a 17.3% higher specificity. |
Mutegeki, 2023
(8) |
Supervised ML,
such as Decision Trees, Random Forest, XGBoost |
Emergency Severity Index (ESI), Patient Medical Data |
The Histogram-Based Gradient Boosting Classifier performed best, achieving a 91% AUC and 70% F1 score |
Hatachi, 2023
(9) |
Supervised ML
(Logistic Regression, Support Vector Machine (SVM), Random Forest, Extreme Gradient Boosting (XGBoost)) |
Hospital Admission Status, Age, Vital Signs, Symptoms |
Yes |
Choi, 2023
(10) |
Supervised ML
(XGBoost) |
Vital signs, Mental status, Laboratory results, Electrocardiograms (ECGs) |
Yes, it could predict the need for Intubation, Admission to the Intensive Care Unit (ICU), Administration of inotropes or vasopressors, and In-hospital cardiac arrest |
Aljubran, 2023
(11) |
Supervised ML |
Emergency patient records (retrospective) |
Yes, it can aid in prioritizing care and predicting outcomes |
Chen, 2023
(12) |
EE-Explorer system
(a kind of supervised ML) |
Metadata (events, symptoms, and medical history) and ocular surface images via smartphones |
Yes, but just in eye emergencies |
Peng, 2023
(13) |
Image-Based AI, Natural Language Processing (NLP) using ChatGPT, Supervised Learning |
Medical History, Images captured using slit lamp equipment (first stage) and smartphone devices (second stage) |
Yes, but just in eye emergencies |
Gebrael
, 2023
(14) |
ChatGPT |
Demographics (age, ethnicity), Medical history, ER visit details (including pathology types, tumor metastasis, co-existing conditions) |
ChatGPT could be instrumental in triaging patients with metastatic prostate cancer |
Karlafti, 2023
(15) |
Neural network |
Age, Gender |
Yes |
Elhaj, 2023
(16) |
Supervised machine learning techniques |
Body temperature, Respiratory rate, Heart rate, Blood pressure, Oxygen saturation, Chief complaints, Chronic illnesses |
Yes |
Savage, 2024
(17) |
AI systems |
Contrast-enhanced CT (CECT) scans of the chest and abdomen |
Sensitivity: Without AI: 80.0%, With AI: 96.2% (P=0.03)
Specificity: Both phases: 99.9% (P=0.58, no significant difference)
in detecting IPE (Inferior Vena Cava Embolism) |
Tortum, 2024
(18) |
Investigates the effectiveness of three artificial intelligence (AI) models-ChatGPT, Gemini, and Pi |
Primary complaints, Arterial blood pressure, Heart rates, Peripheral oxygen saturation, Body temperature, Age, Gender |
Yes, but:
Under triage rates for ChatGPT:
- 26.5% for yellow-coded patients
- 42.6% for red-coded patients
|
Menshawi, 2024
(19) |
Multi-model machine-learning framework |
Medical Conditions, Demographics, Vital Signs |
Yes |
Hinson, 2024
(20) |
Triage GO (employs machine learning) |
Demographics, Vital signs, and Chief complaints |
Significant reductions in time to emergency cardiovascular procedures |
Pasli, 2024
(21) |
GPT-4 |
Chief complaints, Vital parameters, Medical history |
Yes |
Table 2. The application of machine learning models for mortality prediction
Author |
Technique |
Patients/variables |
Finding |
Chang, 2023
(22) |
CNN-based machine learning model |
ECG |
Using ECG data effectively identifies 30-day mortality risk. |
Jeon, 2023
(23) |
ML (light gradient boosting machine) |
Patients diagnosed with Sepsis |
Outperformed traditional clinical scoring systems |
Yaddaden, 2023
(24) |
Machine Learning-Based Pre-Diagnosis Tools
|
Patients diagnosed with COVID-19 |
It achieved high accuracy: 97% for hospitalization prediction, 86.41% for mortality prediction, and 99.80% for triage acuity prediction. |
Tschoellitsch, 2023
(25) |
Machine learning
|
Patient Presentation |
Admission to Ward Observation: AUC-ROC of 0.842 ± 0.00
Admission to Intensive Care: AUC-ROC of 0.819 ± 0.002
30-Day Mortality Prediction: AUC-ROC of 0.925 ± 0.001 |
Lee, 2023
(26) |
AI model
|
Age, Sex, Intentionality, Injury, Emergent symptoms, AVPU scale (Alert/Verbal/Painful/Unresponsive), Korean Triage and Acuity Scale (KTAS), and Vital signs |
Significantly enhance mortality prediction for ED patients. |
Discussion
Based on the findings summarized in the text, ML and AI techniques demonstrate strong potential in enhancing patient risk stratification and outcome prediction in emergency care settings. CNN-based models effectively utilize ECG data to predict 30-day mortality risk, while light gradient boosting machines outperform traditional clinical scoring systems in sepsis prognosis. Pre-diagnosis ML tools achieve high accuracy in predicting hospitalization, mortality, and triage acuity for COVID-19 patients (24).
Additionally, ML models show robust performance in predicting admissions in wards or intensive care units and 30-day mortality based on patient presentation data (25). AI models incorporating demographic, clinical, and triage variables significantly improve mortality prediction for ED patients. Collectively, these approaches highlight the value of integrating diverse patient data with advanced ML algorithms to improve clinical decision-making and patient outcomes in emergency settings.
Mortality Prediction
Recent studies have explored the application of ML techniques in emergency medicine, focusing on mortality prediction and triage acuity. Chang et al. (22) conducted an original research study involving 1,200 patients, utilizing 12-lead electrocardiogram data to predict acute mortality. Similarly, Jeon et al. (23) performed a retrospective cohort study with 800 patients, investigating mortality prediction in sepsis cases based on Sepsis-3 definitions. Yaddaden et al. (24) presented findings at a conference, estimating a sample of 1,500 patients, and discussed ML-based pre-diagnosis tools for predicting hospitalization, mortality, and triage acuity. Tschoellitsch et al. (25) also conducted original research with 1,000 patients, focusing on the integration of triage data into ML models for improved admission and mortality predictions.
Lastly, Lee et al. (26) utilized a retrospective cohort study involving 600 patients to develop an AI model aimed at predicting trauma mortality. Collectively, these studies underscore the diverse methodologies and patient populations employed in ML applications within EDs, contributing to enhanced predictive analytics in clinical practice.
Among the 12 articles reviewed (10-21), approximately 36% utilized demographic data, while the majority of studies measured vital signs, including blood pressure, oxygen saturation, pulse rate, and temperature. Additionally, around 30% of the studies incorporated patient complaints into their analysis. This indicates a varied approach to incorporating demographic and clinical data across the studies, reflecting different focuses and methodologies in the use of ML for emergency medicine triage and decision support.
Staff Burnout
Eugennia et al. (27) explored how AI is applied in patient triage in emergency services, focusing on optimizing response time and resource allocation. The review highlighted that AI improves triage efficiency, reduces errors in classification, and aids in identifying critical outcomes, particularly during high-demand situations, such as the COVID-19 pandemic. However, challenges include resistance from healthcare professionals and integration with existing systems. The study concludes that while AI enhances diagnostic accuracy and resource management, overcoming cultural and operational barriers and ensuring ethical guidelines and continuous training are crucial for successful implementation in emergency care.
ML vs E-triage
Recent studies, such as those conducted by Levin et al. (28), have indicated that electronic triage (E-triage) demonstrates a greater accuracy in classifying patients at Emergency Severity Index (ESI) level 3, utilizing remotely collected data. ML models, including Gradient Boosting, Random Forest, and CatBoost, have been developed to predict clinical characteristics in EDs. These models incorporate both structured data (e.g., blood pressure, pulse rate, and respiratory rate) and unstructured data (e.g., patient complaints), which are processed through NLP techniques to enhance predictive accuracy (29). However, E-triage systems often rely on the remote assessment of patient demographics and health status to inform triage decisions (30). Despite the advancements in ML capabilities, E-triage systems may encounter performance challenges for various reasons. Notably, recent studies from 2023 and 2024 have increasingly favored the application of ML over traditional E-triage methodologies.
Another study examined the practical implications of AI-driven triage, including improved patient management, reduction of human error through minimized reliance on human operators, efficient resource allocation using algorithms, and data-driven decision making by utilizing historical emergency room visit data. The study suggests that AI may enhance triage accuracy and holds potential for broader applications (29).
Challenges of AI in ED
One significant challenge is the need for wide-ranging staff training, as healthcare staff must be prepared with the necessary skills to effectively use AI tools and interpret their outputs. This training is central not only for enhancing the technical ability of the staff but also for developing trust in AI systems, which can be met with cynicism due to concerns about reliability and accuracy (30). Additionally, integrating AI solutions with existing healthcare systems poses another problem. Many emergency departments operate on legacy systems that may not be compatible with new AI technologies, necessitating significant modifications or upgrades to facilitate seamless integration (31). This can lead to troubles in the system and require additional resources to manage the change effectively.
Furthermore, cost considerations are principal; the financial investment required for AI implementation, including software acquisition, system upgrades, and ongoing maintenance, can be substantial. Budget constraints in healthcare settings often limit the ability to invest in such technologies, despite their potential to improve patient outcomes and operational efficiency (32). Therefore, addressing these challenges through strategic planning, adequate funding, and targeted training programs is essential for the successful adoption of AI in emergency departments. The limitation of this study, including the absence of clearly defined inclusion and exclusion criteria, reduced the transparency and rigor of the review process.
Conclusions
ML offers immense potential to enhance patient triage and clinical decision-making by improving accuracy, reducing workload, and enabling early detection of critical conditions. However, addressing challenges related to bias, validation, ethical concerns, and system integration is essential to fully realize its benefits. With ongoing advancements in AI, the future of ML in nursing and emergency care looks promising, potentially transforming healthcare delivery. By the authors’ suggestion, the integration of ML models for the simultaneous interpretation of vital signs, electrocardiograms, and patient chief complaints is recommended as a valuable tool to assist nurses in patient triage and clinical decision-making.
Conflict of Interest
The authors declared that there was no conflict of interest.