³Division of Pulmonology and Critical Care Medicine, Department of Internal Medicine, Korea University Guro Hospital, Korea University College of Medicine, Seoul, Korea

⁴Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Chungnam National University Hospital, Chungnam National University College of Medicine, Daejeon, Korea

⁵Clinical Trial Center, Chungnam National University Hospital, Daejeon, Korea

⁶Division of Pulmonology, Department of Internal Medicine, Chamjoeun Hospital, Gwangju, Korea

⁷Department of Surgery, Chungbuk National University College of Medicine, Cheongju, Korea

Correspondence to: Jae Young Moon, Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Chungnam National University Hospital, Chungnam National University College of Medicine, 282 Munhwa-ro, Jung-gu, Daejeon 35015, Korea Tel: +82-42-280-7142, Fax: +82-42-257-5753, E-mail: diffable@hanmail.net

Received 2016 December 16; Revised 2017 April 17; Accepted 2017 May 24.

Abstract

Background

The Acute Physiology and Chronic Health Evaluation (APACHE) II model has been widely used in Korea. However, there have been few studies on the APACHE IV model in Korean intensive care units (ICUs). The aim of this study was to compare the ability of APACHE IV and APACHE II in predicting hospital mortality, and to investigate the ability of APACHE IV as a critical care triage criterion.

Methods

The study was designed as a prospective cohort study. Measurements of discrimination and calibration were performed using the area under the receiver operating characteristic curve (AUROC) and the Hosmer-Lemeshow goodness-of-fit test respectively. We also calculated the standardized mortality ratio (SMR).

Results

The APACHE IV score, the Charlson Comorbidity index (CCI) score, acute respiratory distress syndrome, and unplanned ICU admissions were independently associated with hospital mortality. The calibration, discrimination, and SMR of APACHE IV were good (H = 7.67, P = 0.465; C = 3.42, P = 0.905; AUROC = 0.759; SMR = 1.00). However, the explanatory power of an APACHE IV score >93 alone on hospital mortality was low at 44.1%. The explanatory power was increased to 53.8% when the hospital mortality was predicted using a model that considers APACHE IV >93 scores, medical admission, and risk factors for CCI >3 coincidentally. However, the discriminative ability of the prediction model was unsatisfactory (C index <0.70).

Conclusions

The APACHE IV presented good discrimination, calibration, and SMR for hospital mortality.

Keywords: APACHE IV; calibration; discrimination; intensive care units; triage.

Introduction

Due to population aging and chronicization of diseases, the severity of diseases among patients admitted to intensive care units (ICUs) has been increasing. Treatment costs have also been increasing for these patients. Since ICUs have limited medical resources, an objective patient triage protocol and evaluation of the severity of a condition must be developed to minimize unnecessary use of ICUs and allow for efficient use of the limited healthcare resources.

Prognostic scoring systems have been developed to estimate the in-hospital mortality of ICU patients [1-4]. For this reason, over the last three decades, severity scoring systems, such as the Acute Physiology and Chronic Health Evaluation (APACHE) score, the Simplified Acute Physiology Score (SAPS), and the Mortality Probability Model (MPM), have been attempted to be used as a critical care triage criterion beyond predicting hospital mortality in critically ill patients [5].

The APACHE scoring system developed by Knaus et al. [1] achieved higher calibration than Zimmerman et al. [6] after a series of improvements, suggesting that APACHE IV is more accurate [3,7]. According to recent studies, APACHE IV exhibits satisfactory discriminatory performance both in the United States, where it was first developed, and outside the United States [8-11]. The older APACHE II model has been validated in Korean populations; it exhibits poor calibration and modest discrimination for hospital mortality [12]. Whereas, the performance of the APACHE IV has not been sufficiently examined.

This study aimed to investigate the suitability of APACHE IV severity scores and MPMs in an ICU within a tertiary general hospital, by analyzing the relationships among APACHE IV scores at the time of the admission, the predicted mortality rate, and the actual mortality rate comparing with APACHE II model. This study also verified the usefulness of the APACHE IV score as a standard triage protocol for admission in the ICU.

Materials and Methods

1) Patients

The study was conducted in a 1,200-bed capacity referral hospital with four adults’ ICU: medical, surgical, emergency, and cardiac ICU, respectively. Two dedicated intensivists supervised all of 56 ICU beds in all ICUs on a semi-closed system. All patients were admitted between August 1, 2013 and July 31, 2014. The same type of patients suggested by Zimmerman et al. [6] for the development of APACHE IV were involved in this study. We excluded patients if they were younger than 17 years old or if the primary outcome of hospital mortality was uncertain. We also excluded patients with an ICU stay <48 hours, patients with burns, patients missing an APACHE IV score on day 1 in the ICU, patients who were admitted for simple postcardiovascular intervention monitoring, and patients who gave do-not-resuscitate orders. Three hundred and sixty-four patients were screened during the study period and 318 eligible patients were enrolled for analysis (Figure 1). This study was approved by the institutional review board of Chungnam National University Hospital (No. 2015-06-053), in accordance with the Declaration of Helsinki.

Figure 1.

Flow chart of the study population. Initially, 364 intensive care unit patients were enrolled from August 1, 2013 to July 31, 2014. The following patients were excluded: patients who were being readmitted (n = 2), patients who had missing data (n = 8), pediatric patients (n = 1), duplicated data (n = 7), and patients who lost to follow up (n = 28).

2) Differences between APACHE II and APACHE IV model

APACHE II score is calculated based on 12 physiologic criteria and estimates risk based on data available within the first 24 hours of an ICU stay [13,14]. APACHE IV was designed to assess the severity of illness as well as the prognosis in the ICU and has 17 physiological criteria, adding new variables such as mechanical ventilation, thrombolysis, impact of sedation on Glasgow Coma Scale, rescaled Glasgow Coma Scale, PaO₂/ FiO₂ ratio and disease-specific subgroups, to the existing APACHE III variables [6]. Disease-specific scoring systems have been developed for several important subgroups treated in the ICU since an APACHE III model [15].

3) Data collection

Two critical care fellows and one trained nurse prospectively collected and reviewed electronic medical records. The electronic medical records provided all of the data required to predict the mortality rate using APACHE II and APACHE IV. Predicted hospital mortalities were calculated using the equations of APACHE II and APACHE IV as follows: logit for APACHE II = –3.517 + (APACHE II) × 0.146. The APACHE IV score and predicted mortality rate calculation on a website (http://www.mecriticalcare.net/icu_scores/apacheIV.php) were used in the present study.

4) Identification of risk factors independent of the APACHE IV score and triage model development

Cox proportional hazards regression was conducted to examine associations with death after adjustment for the APACHE IV score. Hazard ratios (HRs) were used to quantify the relationship between risk factors and death. An ICU triage model, which predicts hospital mortality, was constructed by combining the APACHE IV score and the other risk factors identified above the Cox regression models.

5) Statistical analysis

Descriptive statistics are presented as medians and interquartile ranges, or as numbers with percentages. A univariate logistic regression analysis was performed to evaluate any associations between various risk factors and hospital mortality. Among the variables used in the model, the risk factors with P-values less than 0.05 were selected for multivariate logistic regression analysis.

Discrimination is defined as the power to distinguish between survivors and non-survivors, and this was evaluated by receiver operating characteristic (ROC) analysis [6]. Calibration was defined as agreement between individual probabilities and actual outcomes. It was assessed using the Hosmer-Lemeshow goodness-of-fit C statistic with P-values greater than 0.05 indicating good calibration [7]. The standardized mortality ratio (SMR) was the ratio between the observed and predicted number of deaths. To test for statistical significance, we calculated 95% confidence interval (CI) according to the method described by Hosmer and Lemeshow [6]. We estimated HRs in univariate Cox proportional hazards models with 95% CIs and level of statistical significance. To test the discrimination ability of different combinations of parameters like APACHE IV, Charlson Comorbidity index (CCI), and department, we used the area under the receiver operating characteristic curve (AUROC) by Delong method.

Statistical analysis was performed using SPSS version 21.0 (IBM Corp., Armonk, NY, USA). STATA version 12.0 (StataCorp., College Station, TX, USA) was used for C-statistics and the R and MKmisc version 3.2.2 was used for H-statistics.

Results

1) Baseline characteristics of the population

The baseline characteristics of the patients are shown in Table 1. The total number of patients was 318, 79 of whom were non-survivors, exhibiting a 24.8% mortality rate. Among the mortality factors, age, CCI scores, unplanned ICU admission, use of vasoactive agents, acute respiratory distress syndrome (ARDS), severe sepsis or septic shock, APACHE II scores, and APACHE IV scores exhibited statistically significant differences between the survivors and non-survivors (P < 0.001). The mean age of the non-survivors was 70.7 ± 12.6 years, which was higher than that of the survivors (63.1 ± 15.5 years). The mean CCI score of the non-survivors was 5.4 ± 3.2 points, exceeding that of the survivors (3.4 ± 2.5 points), suggesting that the non-survivors had more comorbidities than the survivors. No statistically significant differences were found for sex, body mass index, and ICU lengths of stay between the survivors and non-survivors.

Table 1.

Patient characteristics, scores, and predicted mortality in each prognostic model

2) Performance of the APACHE IV and APACHE II models in the prediction of hospital mortality

Both prognostic models showed reasonable discrimination and calibration (Table 2). The AUROC of the APACHE IV and APACHE II models were 0.759 and 0.752. ROC curves for the two scoring systems are shown in Figure 2. Hosmer-Lemeshow goodness-of-fit test generated P-values >0.05 for both APACHE II and APACHE IV, indicating that the models were comparable. APACHE IV exhibited the same mortality prediction rate as the observed mortality. The APACHE II model exhibited a lower SMR than the APACHE IV model (SMR APACHE IV, 1.000 [95% CI, 0.789 to 1.250]; APACHE II, 0.991 [95% CI, 0.788 to 1.248]).

Table 2.

Discrimination and calibration of APACHE IV and APACHE II

Figure 2.

Comparison of the area under the receiver operating characteristic curves of APACHE II and APACHE IV. The areas under the receiver operating characteristic curve were 0.759 and 0.752 in APACHE IV and APACHE II, respectively. APACHE: Acute Physiology and Chronic Health Evaluation.

3) Performance characteristics of different combination of parameters for predicting hospital mortality

The multivariate logistic regression model was used to determine independent risk factors for hospital mortality by using all variables with a P-value <0.05 in the univariate model. Among the variables used in the analysis, the APACHE IV score (OR, 1.023; P < 0.001; 95% CI, 1.012 to 1.034), the CCI (OR, 1.200; P < 0.001; 95% CI, 1.053 to 1.368), ARDS (OR, 13.187; P < 0.001; 95% CI, 3.941 to 44.122) and unplanned ICU admission (OR, 2.239; P = 0.015; 95% CI, 1.169 to 4.287) were associated independently with hospital mortality (Table 3).

Table 3.

Independent risk factors for hospital mortality

With a cutoff score of 93, the APACHE IV score predicted hospital mortality with the highest sensitivity and specificity. The highest sensitivity and specificity were observed for CCI scores greater than 3. Table 4 shows that the HRs for hospital mortality. Medical admission, ARDS, CCI >3, and an APACHE IV >93 score were significant risk factors for hospital mortality.

Table 4.

Predicting value for hospital mortality in patients assessed by Cox proportion hazards models

The explanatory power of APACHE IV scores >93 in predicting the hospital mortality rate was 44.1%; the model explained 44.1% of the total variance. With regards to models in which medical admission, which is a risk factor for hospital mortality, CCI scores >3, and APACHE IV scores >93 were added, the APACHE IV scores >93 and medical admission model had an explanatory power of 48.9%, and the APACHE IV scores >93, medical admission, and CCL >3 model had an explanatory power of 53.8%. Therefore, including risk factors in the models improved their explanatory power. However, the discriminative ability of the prediction models was not satisfactory to use them in a triage protocol for admission in the ICU, since all three models had a C index lower than 0.7 (Table 5).

Table 5.

Performance characteristics of different combination of parameters for predicting hospital mortality in patients

Discussion

The present study aimed to investigate the suitability of APACHE IV severity scores and MPMs in the ICU of a tertiary general hospital. The study also evaluated the usefulness of APACHE IV scores as a single criterion of a triage protocol for admission in ICUs.

There has been much debate on the need for objective directives to follow for an ICU admission triage, which aims to efficiently provide critically ill patients with resources within ICUs [16-18]. A substantial amount of research has also been conducted on the efficacy of existing physiological scores in predicting mortality in ICUs [19-22]. Previous studies have proposed various triage protocols for ICU admission [23].

Since the APACHE scoring system is based on objective physiological factors, it eliminates the possibility of errors made by the user. It also allows for simultaneous comparison and prospective analyses of patients from different ICUs, and can be applied to a wide diversity of patients. Therefore, it was our hypothesis that the APACHE scoring system would be useful as the admission triage criterion.

This study showed that APACHE IV had good calibration and modest discrimination among the critically ill patients in a single center. Moreover, APACHE IV showed an SMR close to the actual mortality rate. Therefore, the suitability of APACHE IV for evaluating the severity of patients’ conditions and predicting their prognoses was verified in this urban referral hospital. It was also found that APACHE IV makes more accurate predictions of patients’ prognoses compared to APACHE II scores in even single center. This could be explained due to the advance of APACHE IV model using additional factors such as mechanical ventilation support, disease specific subgroup analysis, and the specific reason for ICU admission. Daley et al. [24] pointed out that APACHE II has been widely used for measuring ICU performance but this scoring system is not disease specific [25-29]. The same as that APACHE IV scoring system show more reliable prediction in Asia population were observed in other validation studies [30-33].

In aspect of the discriminative ability of new prediction models, the result exhibited unsatisfactory discrimination to use them in a triage protocol for admission in the ICU, as showing all three models had a C index lower than 0.7. There are some reasons why the new prediction model showed unsatisfactory discrimination as an ICU triage protocol. First, the semi-closed system of the ICUs in which the present study was conducted would have prevented lots of patients with a too-sick-to-benefit status from admitting to ICUs in advance by intensivists [34]. Second, the sample size was smaller than that of previous studies on triage models leading to insufficient statistical analysis [35]. Finally, despite establishing 93 as our cutoff score for APACHE IV, the higher score of a prognostic model has greater explanatory power. Though, the present study was limited to comparing the scores divided into only two ranges (<93 or ≥93).

This study had some strengths. The validity of APACHE IV in surgical and medical ICUs, emergency ICUs, and cardiovascular ICUs was evaluated by using data of the same type of patients suggested by Zimmerman et al. [6] for the development of APACHE IV. Second, the present study is meaningful in that it attempted to verify the validity of a standard prognostic scoring system for few domestic studies. Moreover, the results suggest that the APACHE IV system makes more accurate predictions of patients’ prognoses compared to APACHE II scores even in single centers. Finally, within the scope what we know, there is rare research demonstrating that the APACHE IV model would not be advisable as a single criterion for admission in Korea ICU.

However, the study results may contain selection bias since this study took place in only one institution. We could not obtain information on the survival status of patients who could not be monitored due to loss to follow up. And this study had a smaller sample size of 318 patients compared to the sample sizes used in previous studies [19,20,36].

According to our results, in order to establish objective criteria for admission to ICUs, overall clinical judgment of internal medicine patients, severity of comorbidities, main diagnosis type (ARDS, septic shock), age, likelihood of recovery, opinions of medical professionals, patients’ conditions, and prognoses may be helpful. And multilateral research involving larger patient populations and disease groups is essential.

In conclusion, in this work, the APACHE IV scoring system exhibits satisfactory discrimination and excellent calibration, but the result supposed that it was not appropriate to be used as a single criterion for ICU admission. Further research on determining the ICU admission priority of critically ill patients would be necessary.

Notes

No potential conflict of interest relevant to this article was reported.

Acknowledgements

Study design and interpretation of data: Sang IL Park

References

1. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818–29.

2. Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand 2008;52:467–78.

3. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3: from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005;31:1345–55.

4. Lemeshow S, Teres D, Pastides H, Avrunin JS, Steingrub JS. A method for predicting survival and mortality of ICU patients using objectively derived weights. Crit Care Med 1985;13:519–25.

5. Caldeira VM, Silva Júnior JM, Oliveira AM, Rezende S, Araújo LA, Santana MR, et al. Criteria for patient admission to an intensive care unit and related mortality rates. Rev Assoc Med Bras (1992) 2010;56:528–34.

6. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 2006;34:1297–310.

7. Le Gall JR, Loirat P, Alperovitch A, Glaser P, Granthil C, Mathieu D, et al. A simplified acute physiology score for ICU patients. Crit Care Med 1984;12:975–7.

8. Brinkman S, Bakhshi-Raiez F, Abu-Hanna A, de Jonge E, Bosman RJ, Peelen L, et al. External validation of Acute Physiology and Chronic Health Evaluation IV in Dutch intensive care units and comparison with Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II. J Crit Care 2011;26:105.e11–8.

9. Keegan MT, Gajic O, Afessa B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest 2012;142:851–8.

10. Nassar Junior AP, Mocelin AO, Andrade FM, Brauer L, Giannini FP, Nunes AL, et al. SAPS 3, APACHE IV or GRACE: which score to choose for acute coronary syndrome patients in intensive care units? Sao Paulo Med J 2013;131:173–8.

11. Berger MM, Marazzi A, Freeman J, Chioléro R. Evaluation of the consistency of Acute Physiology and Chronic Health Evaluation (APACHE II) scoring in a surgical intensive care unit. Crit Care Med 1992;20:1681–7.

12. Kim JY, Lim SY, Jeon K, Koh Y, Lim CM, Koh SO, et al. External validation of the Acute Physiology and Chronic Health Evaluation II in Korean intensive care units. Yonsei Med J 2013;54:425–31.

13. Gupta R, Arora VK. Performance evaluation of APACHE II score for an Indian patient with respiratory problems. Indian J Med Res 2004;119:273–82.

14. Ludwigs U, Csatlos M, Hulting J. Predicting in-hospital mortality in acute myocardial infarction: impact of thrombolytic therapy on APACHE II performance. Scand Cardiovasc J 2000;34:371–6.

15. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, et al. The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100:1619–36.

16. Sprung CL, Danis M, Iapichino G, Artigas A, Kesecioglu J, Moreno R, et al. Triage of intensive care patients: identifying agreement and controversy. Intensive Care Med 2013;39:1916–24.

17. Orsini J, Butala A, Ahmad N, Llosa A, Prajapati R, Fishkin E. Factors influencing triage decisions in patients referred for ICU admission. J Clin Med Res 2013;5:343–9.

18. Orsini J, Blaak C, Yeh A, Fonseca X, Helm T, Butala A, et al. Triage of patients consulted for ICU admission during times of ICU-bed shortage. J Clin Med Res 2014;6:463–8.

19. Lee H, Shon YJ, Kim H, Paik H, Park HP. Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit. Korean J Anesthesiol 2014;67:115–22.

20. Sedloň P, Kameník L, Škvařil J, Malý M, Táborský M, Zavoral M. Comparison of the accuracy and correctness of mortality estimates for intensive care unit patients in internal clinics of the Czech Republic using APACHE II, APACHE IV, SAPS 3 and MPMoIII models. Med Glas (Zenica) 2016;13:82–9.

21. Salluh JI, Soares M. ICU severity of illness scores: APACHE, SAPS and MPM. Curr Opin Crit Care 2014;20:557–65.

22. Costa e Silva VT, de Castro I, Liaño F, Muriel A, Rodríguez-Palomares JR, Yu L. Performance of the third-generation models of severity scoring systems (APACHE IV, SAPS 3 and MPM-III) in acute kidney injury critically ill patients. Nephrol Dial Transplant 2011;26:3894–901.

23. Aminiahidashti H, Bozorgi F, Montazer SH, Baboli M, Firouzian A. Comparison of APACHE II and SAPS II scoring systems in prediction of critically ill patients’ outcome. Emerg (Tehran) 2017;5e4.

24. Daley J, Jencks S, Draper D, Lenhart G, Thomas N, Walker J. Predicting hospital-associated mortality for Medicare patients: a method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. JAMA 1988;260:3617–24.

25. Kopterides P, Liberopoulos P, Ilias I, Anthi A, Pragkastis D, Tsangaris I, et al. General prognostic scores in outcome prediction for cancer patients admitted to the intensive care unit. Am J Crit Care 2011;20:56–66.

26. Gilani MT, Razavi M, Azad AM. A comparison of Simplified Acute Physiology Score II, Acute Physiology and Chronic Health Evaluation II and Acute Physiology and Chronic Health Evaluation III scoring system in predicting mortality and length of stay at surgical intensive care unit. Niger Med J 2014;55:144–7.

27. Kramer AA, Higgins TL, Zimmerman JE. Comparison of the Mortality Probability Admission Model III, National Quality Forum, and Acute Physiology and Chronic Health Evaluation IV hospital mortality models: implications for national benchmarking. Crit Care Med 2014;42:544–53.

28. Juneja D, Singh O, Nasa P, Dang R. Comparison of newer scoring systems with the conventional scoring systems in general intensive care population. Minerva Anestesiol 2012;78:194–200.

29. Xing X, Gao Y, Wang H, Huang C, Qu S, Zhang H, et al. Performance of three prognostic models in patients with cancer in need of intensive care in a medical center in China. PLoS One 2015;10e0131329.

30. Wong RS, Ismail NA, Tan CC. An external independent validation of APACHE IV in a Malaysian intensive care unit. Ann Acad Med Singapore 2015;44:127–32.

31. Bhattacharyya M, Todi S. APACHE IV: benchmarking in an Indian ICU. Crit Care 2009;13(Suppl 1):P510.

32. Yamin S, Vaswani AK, Afreedi M. Predictive efficasy of APACHE IV at ICU.s of CHK. Pak J Chest Med 2011;17

33. Ayazoglu TA. A comparison of APACHE II and APACHE IV scoring systems in predicting outcome in patients admitted with stroke to an intensive care unit. Anaesth Pain Intensive Care 2011;15:7–12.

34. Garrouste-Orgeas M, Montuclard L, Timsit JF, Misset B, Christias M, Carlet J. Triaging patients to the ICU: a pilot study of factors influencing admission decisions and patient outcomes. Intensive Care Med 2003;29:774–81.

35. Adeniji KA, Cusack R. The Simple Triage Scoring System (STSS) successfully predicts mortality and critical care resource utilization in H1N1 pandemic flu: a retrospective analysis. Crit Care 2011;15:R39.

36. Ho KM, Williams TA, Harahsheh Y, Higgins TL. Using patient admission characteristics alone to predict mortality of critically ill patients: a comparison of 3 prognostic scores. J Crit Care 2016;31:21–5.

Article information Continued

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Table 1.

Patient characteristics, scores, and predicted mortality in each prognostic model

Variable	Total (n = 318)	Survivor (n = 239)	Non-survivor (n = 79)	P-value
Age (yr)		63.1 ± 15.5	70.7 ± 12.6	<0.001
Male sex		131 (54.8)	49 (62.0)	0.262
Body mass index (kg/m²)		22.6 ± 3.9	22.6 ± 4.2	0.948
Charlson Comorbidity index score, age adjusted		3.4 ± 2.5	5.4 ± 3.2	<0.001
Route of admission	315			0.001
Ward		68 (29.2)	39 (50.6)
Emergency room		168 (70.0)	38 (49.4)
Other ICU/hospital		2 (0.9)	0
Admission type	305			0.092
Emergency surgery		30 (13.1)	9 (11.8)
Elective surgery		47 (20.5)	5 (6.6)
No surgery		152 (66.4)	62 (81.6)
Unplanned ICU admission	307	79 (34.6)	47 (59.5)	<0.001
Comorbidities
Heart failure		22 (9.2)	3 (3.8)	0.122
Solid cancer		16 (6.7)	4 (5.1)	0.791
Chronic pulmonary disease		11 (4.6)	8 (10.1)	0.097
Infection-related admission	292	28 (13.0)	16 (20.8)	0.103
Mechanical ventilation (invasive)		74 (31.0)	35 (44.3)	0.030
CRRT		18 (7.5)	12 (15.2)	0.043
Vasoactive agent		52 (21.8)	42 (53.2)	<0.001
Major diagnosis
ARDS		5 (2.1)	19 (24.1)	<0.001
Severe sepsis or septic shock		33 (13.8)	34 (43.0)	<0.001
CPCR survivor		15 (6.3)	10 (12.7)	0.068
APACHE II score		21.0 ± 8.7	29.4 ± 8.8	<0.001
APACHE II predicted mortality		46.0 ± 11.1	56.6 ± 11.0	<0.001
APACHE IV score		71.4 ± 32.6	105.3 ± 37.8	<0.001
APACHE IV predicted mortality		24.5 ± 24.2	50.6 ± 30.1	<0.001
ICU length of stay (d)		11.0 ± 12.8	12.5 ± 12.2	0.367
Hospital length of stay (d)		41.7 ± 47.4	39.0 ± 89.2	0.736

Values are presented as mean ± standard deviation or number (%).

ICU: intensive care unit; CRRT: continuous renal replacement therapy; ARDS: acute respiratory distress syndrome; CPCR: cardiopulmonary cerebral resuscitation; APACHE: Acute Physiology and Chronic Health Evaluation.

Model	No.^a	AUROC (95% CI)	Hosmer-Lemeshow goodness-of-fit test				SMR
Model	No.^a	AUROC (95% CI)	C-test	P-value	H-test	P-value	SMR
APACHE IV	304	0.759 (0.699–0.819)	3.42	0.905	7.679	0.465	1.000 (0.789–1.250)
APACHE II	304	0.752 (0.692–0.811)	4.55	0.805	7.817	0.452	0.991 (0.788–1.248)

Variable	Hospital mortality
Variable	Odds ratio	P-value^a
APACHE IV score	1.023 (1.012–1.034)	<0.001
Age	1.015 (0.989–1.045)	0.223
CCI score	1.200 (1.053–1.368)	<0.001
Mechanical ventilation	0.534 (0.254–1.120)	0.097
Vasoactive agent	1.697 (0.798–3.610)	0.169
Sepsis	1.609 (0.729–3.550)	0.239
ARDS	13.187 (3.941–44.122)	<0.001
Unplanned ICU admission	2.239 (1.169–4.287)	0.015

Risk factor	Hazard ratio (95% CI)	P-value^a
Multivariate analysis
Surgical department admission	Reference
Medical department admission	3.534 (1.634–7.692)	0.001
Acute respiratory distress syndrome	2.661 (1.526–4.640)	0.001
CCI score >3	1.819 (1.061–3.120)	0.030
APACHE IV score >93	2.140 (1.304–3.510)	0.003

Model	C-index (95% CI)	P-value
APACHE IV >93	0.680 (0.626–0.731)	Reference
APACHE IV >93 + medical department	0.686 (0.632–0.737)	0.679
APACHE IV >94 + medical department + CCI >3	0.659 (0.604–0.711)	0.372