Skip Navigation
Skip to contents

ACC : Acute and Critical Care

OPEN ACCESS
SEARCH
Search
Advanced Search
mobile menu button

Articles

Page Path
HOME > Acute Crit Care > Volume 33(3); 2018 > Article
Review Article Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning
Youngnam Lee1, Joon-myoung Kwon2orcid, Yeha Lee1, Hyunho Park1, Hugh Cho1, Jinsik Park2
Acute and Critical Care 2018;33(3):117-120.
DOI: https://doi.org/10.4266/acc.2018.00290
Published online: August 31, 2018

1VUNO, Seoul, Korea

2Department of Emergency Medicine, Mediplex Sejong Hospital, Incheon, Korea

Corresponding author Joon-myoung Kwon Department of Emergency Medicine, Mediplex Sejong Hospital, 20 Gyeyangmunhwa-ro, Gyeyang-gu, Incheon 21080, Korea Tel: +82-32-240-8000 Fax: +82-32-240-8999 E-mail: kwonjm@sejongh.co.kr
*The first two authors contributed equally to this study.
• Received: August 23, 2018   • Accepted: August 24, 2018

Copyright © 2018 The Korean Society of Critical Care Medicine

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

  • 12,829 Views
  • 531 Download
  • 17 Web of Science
  • 19 Crossref
  • 23 Scopus
Full Article

Download PDF Download PDF

  • With the wider adoption of electronic health records, the rapid response team initially believed that mortalities could be significantly reduced but due to low accuracy and false alarms, the healthcare system is currently fraught with many challenges. Rule-based methods (e.g., Modified Early Warning Score) and machine learning (e.g., random forest) were proposed as a solution but not effective. In this article, we introduce the DeepEWS (Deep learning based Early Warning Score), which is based on a novel deep learning algorithm. Relative to the standard of care and current solutions in the marketplace, there is high accuracy, and in the clinical setting even when we consider the number of alarms, the accuracy levels are superior.
  • Keywords: artificial intelligence; cardiac arrest; deep learning; rapid response team
In 2009, the Health Information Technology for Economic and Clinical Health Act was implemented to roll out electronic health records (EHRs) and it was rapidly adopted. A total of 30 million dollars was invested into this policy that incorporated incentives and penalties for widely adopting EHRs. EHRs include demographic information, present and past diagnoses, laboratory results, vital sign, and clinical notes to efficiently manage patients in the clinical setting and can leverage clinical informatics to predict diseases, patient length of stay, and readmission rates.
Due to the variety of clinical information, providers can quickly access through EHRs; a rapid response team (RRT) can effectively reduce the number of in-hospital cardiac arrest and mortalities. However, with the current standard of care (Modified Early Warning Score [MEWS]), the RRT is suffering from low accuracy and false alarms. In terms of the accuracy, the RRT could only detect 30% of unplanned intensive care unit admissions, and the MEWS that the RRT team employs does not even reach an area under the receiver operating characteristic (AUROC) of 0.8 [1-3]. Previous algorithms have focused on accuracy; however, in the clinical environment false alarms are the most important factor when evaluating an algorithm [4,5]. Whenever alarms sound, in real time RRT reconfirm patients and determine intervention. Therefore, if there are too many false alarms, an unnecessary amount of time and costs are wasted.
To overcome the limits of MEWS, many traditional machine learning algorithms have been incorporated but still suffer from low accuracy and false alarms. In this article, we introduce the Deep learning based Early Warning Score (DeepEWS), that is based on a novel deep learning algorithm [6]. Deep learning has demonstrated state of the art results for computer vision, machine translation and has demonstrated many successful results [7-9]. In the medical domain, deep learning has been limited to being applied to medical images. However, recently there have been new applications to EHR and biosignal research.
DeepEWS was compared to rule-based and the most promising traditional machine learning algorithms in the clinical setting. There were many different scenarios on which the results were compared and particularly, the sensitivity versus the number of alarms effectively demonstrated the superiority in accuracy and ease of clinical implementation for DeepEWS.
Both machine learning and deep learning analyze data through self-learning to solve the task or problem at hand (i.e., predict, classify). However, machine learning requires feature engineering, whereas deep learning does not, and this is the essence of why deep learning provides higher accuracy [10]. Feature engineering strives to solve the problem in the model through pattern recognition by sensitizing various scenarios. In this process, domain knowledge and expertise can translate to significantly different results. However, deep learning uses deep neural networks to find these features automatically. As such, even without domain knowledge and specialist expertise, the model can self-learn. The deeper the neural network, the more diverse features are found, and this translates to higher performance and accuracy in results.
In-hospital cardiac arrests alongside with sepsis is a major factor that determines patient safety. In the United States, every year there are 20,900 cardiac arrest patients, and globally the survival rate for cardiac arrest patients is below 20%. It is well documented in the literature, that 80% of cardiac arrest patients experience deterioration in health, 8 hours before the cardiac arrest event, and there have been many attempts to predict this onset. However, the current standard of care suffers from low accuracy and false alarms that renders the clinical implementation and practical use of these technologies challenging.
DeepEWS solved two major shortcomings that existing algorithms suffer from (low accuracy and false alarms) and provided clinically viable solutions. DeepEWS uses all four vital signs measured for 8 hours that can be easily measured in a hospital setting (systolic blood pressure, body temperature, heart rate, and respiratory rate). The details of the method can be confirmed in [6]. As shown in Figure 1, DeepEWS can measure real-time cardiac arrest risks (within 24 hours of the cardiac arrest event) from vital signs in Electronic Medical Records.
The study period of this paper was from June 2010 to July 2017, with 56,076 patients were admitted to the Mediplex Sejong Hospital, and 415 first cardiac arrest occurred. Four forty-eight patients were excluded because they were admitted or discharged outside the study period.
There are two significant results that the study demonstrated: the accuracy level and applicable in a clinical environment. Figure 2 represents the AUROC for DeepEWS and competing algorithms. DeepEWS shows an AUROC of 0.850 relative to MEWS’s 0.603, a 40.9% improvement in accuracy; (currently, the second version of DeepEWS recorded an AUROC of 0.911, a vast improvement from its previous version). Relative to algorithms used in previous research such as random forest (0.780) and logistic regression (0.613), the accuracy levels are markedly higher as well.
Figure 3 shows the sensitivity relative to the number of alarms and demonstrates how effective DeepEWS is in the clinical environment by maintaining high levels of accuracy even at the low number of alarms. In the real-world setting, an RRT is required to cover many patients so keeping a high level of accuracy with a small number of alarms is quintessential. We can see that at points of the lower number of alarms the accuracy level is significantly higher for DeepEWS relative to other competing algorithms. DeepEWS is more accurate with a lower number of alarms. DeepEWS is 42.7% accurate when the number of alarms is equal to 40 (point B); MEWS is 22.7% accurate when the number of alarms is equal to 110 (point C).
This study has two significant implications. First, the first study using deep learning has significantly improved accuracy. Second, it has been shown that DEWS is applicable in a clinical environment by comparing the accuracy according to the number of alarms. A large amount of false alarms is the factor for why MEWS and traditional machine learning algorithms could not be adopted in the clinical environment. DeepEWS predicts cardiac arrest with high accuracy and low false alarms. There are only four vital signs that are used as inputs that can easily be obtained in any clinical environment and augmenting these results with laboratory results, and clinical notes could enhance the accuracy even further.
▪ We introduce the DeepEWS (Deep learning based Early Warning Score), which is based on a novel deep learning algorithm.
▪ Relative to the standard of care and current solutions in the marketplace, there is high accuracy and low false alarms.

Youngnam Lee, Joon-myoung Kwon, Yeha Lee, and Jinsik Park were involved in developing DEWS (Deep learning based Early Warning Score). VUNO provided support in the form of salaries for authors (Youngnam Lee, Yeha Lee, Hyunho Park, and Hugh Cho), but did not provide funding and did not have any additional role. No other conflict of interest relevant to this article was reported.

Figure 1.
DeepEWS (Deep learning based Early Warning Score) operating on data from Electronic Medical Records. MEWS: Modified Early Warning Score; NIBP(S): non-invasive blood pressure (systolic); NIBP(D): non-invasive blood pressure (diastolic); HR: heart rate; RR: respiratory rate; SPO2: blood oxygen saturation; AVPU: alert, verbal, pain, unresponsive.
acc-2018-00290f1.jpg
Figure 2.
The area under the receiver operating characteristic of algorithms for detecting cardiac arrest. DeepEWS: Deep learning based Early Warning Score; MEWS: Modified Early Warning Score.
acc-2018-00290f2.jpg
Figure 3.
The sensitivity according to the number of alarms. The Y-axis indicates sensitivity, and the X-axis means the number of alarms per 1,000 patients in 1 hour (A: 20, B: 40, C: 110, D: 200, E: 330).
acc-2018-00290f3.jpg
  • 1. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet 2005;365:2091-7.ArticlePubMed
  • 2. Trinkle RM, Flabouris A. Documenting rapid response system afferent limb failure and associated patient outcomes. Resuscitation 2011;82:810-4.ArticlePubMed
  • 3. Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med 2016;44:368-74.ArticlePubMedPMC
  • 4. Gao H, McDonnell A, Harrison DA, Moore T, Adam S, Daly K, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med 2007;33:667-79.ArticlePubMed
  • 5. Bell MB, Konrad D, Granath F, Ekbom A, Martling CR. Prevalence and sensitivity of MET-criteria in a Scandinavian University Hospital. Resuscitation 2006;70:66-73.ArticlePubMed
  • 6. Kwon JM, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. J Am Heart Assoc 2018;7:e008678. ArticlePubMedPMC
  • 7. Deo RC. Machine learning in medicine. Circulation 2015;132:1920-30.ArticlePubMedPMC
  • 8. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016;316:2402-10.ArticlePubMed
  • 9. Jha S, Topol EJ. Adapting to artificial intelligence: radiologists and pathologists as information specialists. JAMA 2016;316:2353-4.ArticlePubMed
  • 10. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436-44.ArticlePubMedPDF

Figure & Data

References

    Citations

    Citations to this article as recorded by  
    • External Validation of Deep Learning-Based Cardiac Arrest Risk Management System for Predicting In-Hospital Cardiac Arrest in Patients Admitted to General Wards Based on Rapid Response System Operating and Nonoperating Periods: A Single-Center Study
      Kyung-Jae Cho, Kwan Hyung Kim, Jaewoo Choi, Dongjoon Yoo, Jeongmin Kim
      Critical Care Medicine.2024; 52(3): e110.     CrossRef
    • Evaluation of optimal scene time interval for out-of-hospital cardiac arrest using a deep neural network
      Seung Jae Shin, Hee Sun Bae, Hyung Jun Moon, Gi Woon Kim, Young Soon Cho, Dong Wook Lee, Dong Kil Jeong, Hyun Joon Kim, Hyun Jung Lee
      The American Journal of Emergency Medicine.2023; 63: 29.     CrossRef
    • Short-term load forecasting based on CEEMDAN and Transformer
      Peng Ran, Kun Dong, Xu Liu, Jing Wang
      Electric Power Systems Research.2023; 214: 108885.     CrossRef
    • Artificial Intelligence in Resuscitation: A Scoping Review
      Dmitriy Viderman, Yerkin Abdildin, Kamila Batkuldinova, Rafael Badenes, Federico Bilotta
      Journal of Clinical Medicine.2023; 12(6): 2254.     CrossRef
    • Prediction of Out-of-Hospital Cardiac Arrest Survival Outcomes Using a Hybrid Agnostic Explanation TabNet Model
      Hung Viet Nguyen, Haewon Byeon
      Mathematics.2023; 11(9): 2030.     CrossRef
    • Enhancing breast cancer diagnosis with deep learning and evolutionary algorithms: A comparison of approaches using different thermographic imaging treatments
      Alberto Nogales, Fernando Pérez-Lara, Álvaro J. García-Tejedor
      Multimedia Tools and Applications.2023; 83(14): 42955.     CrossRef
    • Machine learning pre-hospital real-time cardiac arrest outcome prediction (PReCAP) using time-adaptive cohort model based on the Pan-Asian Resuscitation Outcome Study
      Hansol Chang, Ji Woong Kim, Weon Jung, Sejin Heo, Se Uk Lee, Taerim Kim, Sung Yeon Hwang, Sang Do Shin, Won Chul Cha, Marcus Ong
      Scientific Reports.2023;[Epub]     CrossRef
    • Attempting cardiac arrest prediction using artificial intelligence on vital signs from Electronic Health Records
      Bassel Soudan, Fetna F. Dandachi, Ali Bou Nassif
      Smart Health.2022; 25: 100294.     CrossRef
    • BERT Learns From Electroencephalograms About Parkinson’s Disease: Transformer-Based Models for Aid Diagnosis
      Alberto Nogales, Alvaro J. Garcia-Tejedor, Ana M. Maitin, Antonio Perez-Morales, Maria Dolores Del Castillo, Juan Pablo Romero
      IEEE Access.2022; 10: 101672.     CrossRef
    • Artificial intelligence decision points in an emergency department
      Hansol Chang, Won Chul Cha
      Clinical and Experimental Emergency Medicine.2022; 9(3): 165.     CrossRef
    • Development of Prediction Models for Acute Myocardial Infarction at Prehospital Stage with Machine Learning Based on a Nationwide Database
      Arom Choi, Min Joung Kim, Ji Min Sung, Sunhee Kim, Jayoung Lee, Heejung Hyun, Hyeon Chang Kim, Ji Hoon Kim, Hyuk-Jae Chang
      Journal of Cardiovascular Development and Disease.2022; 9(12): 430.     CrossRef
    • Short-Term Load Forecasting Based on Ceemdan and Transformer
      Peng Ran, Kun Dong, Xu Liu, Jing Wang
      SSRN Electronic Journal .2022;[Epub]     CrossRef
    • A survey of deep learning models in medical therapeutic areas
      Alberto Nogales, Álvaro J. García-Tejedor, Diana Monge, Juan Serrano Vara, Cristina Antón
      Artificial Intelligence in Medicine.2021; 112: 102020.     CrossRef
    • A predictive framework in healthcare: Case study on cardiac arrest prediction
      Samaneh Layeghian Javan, Mohammad Mehdi Sepehri
      Artificial Intelligence in Medicine.2021; 117: 102099.     CrossRef
    • Development of Prediction Models for Acute Myocardial Infarction at Prehospital Stage with Machine Learning Based on a Nationwide Database (Preprint)
      Arom Choi, Min Joung Kim, Ji Min Sung, Sunhee Kim, Jayong Lee, Heejung Hyun, Ji Hoon Kim, Hyuk-Jae Chang
      JMIR Medical Informatics.2021;[Epub]     CrossRef
    • Predicting in-hospital mortality in adult non-traumatic emergency department patients: a retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach
      Kuan-Han Wu, Fu-Jen Cheng, Hsiang-Ling Tai, Jui-Cheng Wang, Yii-Ting Huang, Chih-Min Su, Yun-Nan Chang
      PeerJ.2021; 9: e11988.     CrossRef
    • Cardioinformatics: the nexus of bioinformatics and precision cardiology
      Bohdan B Khomtchouk, Diem-Trang Tran, Kasra A Vand, Matthew Might, Or Gozani, Themistocles L Assimes
      Briefings in Bioinformatics.2020; 21(6): 2031.     CrossRef
    • Development of a Real-Time Risk Prediction Model for In-Hospital Cardiac Arrest in Critically Ill Patients Using Deep Learning: Retrospective Study
      Junetae Kim, Yu Rang Park, Jeong Hoon Lee, Jae-Ho Lee, Young-Hak Kim, Jin Won Huh
      JMIR Medical Informatics.2020; 8(3): e16349.     CrossRef
    • Rapid response systems in Korea
      Bo Young Lee, Sang-Bum Hong
      Acute and Critical Care.2019; 34(2): 108.     CrossRef

    • PubReader PubReader
    • ePub LinkePub Link
    • Cite
      CITE
      export Copy
      Close
      Download Citation
      Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.
      Format:
      • RIS — For EndNote, ProCite, RefWorks, and most other reference management software
      • BibTeX — For JabRef, BibDesk, and other BibTeX-specific software
      Include:
      • Citation for the content below
      Deep Learning in the Medical Domain: Predicting Cardiac Arrest Using Deep Learning
      Acute Crit Care. 2018;33(3):117-120.   Published online August 31, 2018
      DOI: https://doi.org/10.4266/acc.2018.00290
      Close
    • XML DownloadXML Download
    Figure
    We recommend
    ACC : Acute and Critical Care
    © The Korean Society of Critical Care Medicine.