The Best Prediction Model for Trauma Outcomes of the Current Korean Population: a Comparative Study of Three Injury Severity Scoring Systems

Article information

Acute Crit Care. 2016;31(3):221-228

Publication date (electronic) : 2016 August 30

doi : https://doi.org/10.4266/kjccm.2016.00486

Kyoungwon Jung, M.D., Ph.D.¹, John Cook-Jong Lee, M.D., Ph.D.¹, Rae Woong Park, M.D., Ph.D.², Dukyong Yoon, M.D., Ph.D.², Sungjae Jung, B.S.², Younghwan Kim, M.D.¹, Jonghwan Moon, M.D.¹, Yo Huh, M.D.¹, Junsik Kwon, M.D.^,1

¹Division of Trauma Surgery, Department of Surgery, Ajou University School of Medicine, Suwon, Korea

²Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Korea

Correspondence to: Junsik Kwon, Division of Trauma Surgery, Department of Surgery, Ajou University School of Medicine, 206 World cup-ro, Yeoungtong-gu, Suwon 16499, Korea Tel: +82-31-219-7491, Fax: +82-31-219-7765 E-mail: aquaestel@gmail.com

Received 2016 May 31; Revised 2016 July 25; Accepted 2016 July 29.

Abstract

Introduction

Trauma is a worldwide health problem, but its burden is skewed more heavily in developing or underdeveloped countries.[1] Disparity in regard to trauma in developing countries is due to a variety of factors, including lack of approved trauma centers, lack of highly trained medical professionals, and insufficient experience with trauma assessment tools. The prognosis of trauma patients differs greatly among countries with and without a well-organized trauma management system; a study that compared the mortality rate of patients with an Injury Severity Score (ISS) ≥ 15 in low-income versus high-income countries found that the mortality rate was approximately six-fold higher in low-income countries.[2] Such an enormous difference impedes the global standardization of predicting the prognosis of trauma patients and establishment of criteria to fairly assess a country’s trauma system.[3] Korea is a socioeconomically developed country that recently made progress in its health care system. However, a recent survey[4] showed a mortality rate of 35.2% among trauma patients whose deaths were preventable; thus, in terms of trauma management, Korea is similar to a developing country. Korea is working on establishing an organized trauma system to decrease the morality rate of trauma patients whose deaths are preventable to 20% or lower by 2020. An injury severity scoring system (ISSS) will be created to quantify the severity of trauma to compare and predict outcomes across different trauma centers and to quantitatively determine whether Korea’s trauma system should be improved.[5] Since 1971, when an ISSS called the Abbreviated Injury Scale (AIS)[6] was developed in the United States, several scoring systems have been developed. In Korea, traumatology is in the beginning stage of development compared to other advanced countries. A few studies have been conducted on an ISSS in the Korean population. However, these studies included a small sample size[7] or a survey of motor vehicle accident victims.[8] It would be of great significance to evaluate whether representative ISSSs can be preliminarily used in Korea before an appropriate scoring system is created for the Korean population. Therefore, we assessed three representative ISSSs, i.e., ISS, Revised Trauma Score (RTS), and Trauma and Injury Severity Score (TRISS), in terms of ability to predict the outcome of trauma treatment.

Materials and Methods

Results

During the study period, 9,142 patients received inpatient treatment from the Department of Trauma Surgery; of those, 7,120 (6,668 [93.7%] with blunt injuries and 452 [6.3%] with penetrating injuries) met the inclusion criteria and were analyzed. The majority of patients were men (4,826; 67.8%), and mean patient age was 48.85 ± 19.25 years. Overall, 3,402 (47.8%) patients were transferred from another hospital. The respiratory rate per minute was ≥ 30 breaths or ≤ 5 breaths in 342 (2.8%) patients. The systolic blood pressure (SBP) was < 90 mmHg in 485 (6.8%) patients. The GCS score was ≤ 8 in 721 (10.1%) patients (mean GCS score, 13.7 ± 3.2). The most common type of injury was blunt (6668), and traffic accidents were most prevalent, accounting for 40.2% of submitted patients (Table 1). Five hundred sixty-two patients died, with an overall mortality rate of 7.9%. The mean values of ISS, RTS, and TRISS (survival probability) were 11.6 ± 10.4, 7.34 ± 1.53, and 0.92 ± 0.19, respectively. The AUCs for the predictive ability of the ISS, RTS, and TRISS were 0.866, 0.894, and 0.942, respectively (Table 2). The predictive ability of the TRISS was significantly better than those of the ISS (p < 0.001) and RTS (p < 0.001; Table 3, Fig. 1).

Table 1.

Characteristics of the study population

Table 2.

Areas under the ROC curve for the predictive ability of each injury severity scoring system

Table 3.

Comparison of the ISS, RTS, and TRISS in terms of predicting mortality

Fig. 1.

Receiver operating characteristic curves of Injury Severity Score (ISS), Revised Trauma Score (RTS), and Trauma and Injury Severity Score (TRISS) for predicting mortality in the study population.

The cut-off values of each scoring system are shown in Table 4. The cut-off value of the ISS was 15, with a sensitivity of 86.7% and specificity of 74.6%; mortality was predicted with an accuracy of 75.5%. The positive predictive value was low (22.6%), whereas the negative predictive value was very high (98.5%). The cut-off value of the RTS was 7.6954, with a sensitivity of 83.5% and specificity of 88.6%; mortality was predicted with an accuracy of 88.2%. The positive predictive value was 38.5%, which was higher than that of the ISS, and the negative predictive value was 98.4%, which was similar to that of the ISS. The cut-off value of the TRISS was 0.9082, with a sensitivity of 81.9% and specificity of 92.0%; mortality was predicted with an accuracy of 91.2%. Overall, the TRISS had the best results among the three ISSSs; its positive predictive value was the highest at 46.8%.

Table 4.

Cut-off value of each scoring system at the maximum sensitivity and specificity and the accuracy, positive, and negative predictive values of the receiver operating characteristic curve

Discussion

It is critical to establish an ideal trauma scoring system due to its many uses, including predicting the prognosis of trauma patients, comparing treatment methods, using it as a triage tool during transport to a hospital or during transfer between hospitals, using it as a tool for quality management of trauma centers, using it as a preventive program, and using it as a research tool in trauma studies.[12] We used scoring systems that have been developed and widely utilized in countries in which traumatology has progressed, including Korea, to investigate the benefits and limitations of ISSSs. More blunt injuries than penetrating injuries occur in Korea compared to the United States because the use of guns is restricted. Thus, it is important to evaluate the applicability and outcome prediction ability of existing ISSSs primarily developed and widely used in the United States in the Korean patient population. To establish a countrywide trauma system, Korea has established > 10 regional trauma centers and supported them with tax revenues, but it does not yet have a program or specific plans for quality improvement of the centers’ operation. The Korean government has set a goal to reduce the mortality rate of preventable deaths to < 20% by 2020. However, the current mortality rate of preventable deaths, which was the baseline used to set the goal, was determined in a subjective panel review; thus, the objectivity has been debated. Existing research and research efforts are lacking on the topic of ISSSs, so this debate continues. Currently, a trauma database is being established at medical facilities selected as regional trauma centers, but it is not available to general researchers because the reliability of the data is low; furthermore, no attempt has been made to establish Korean norms, such as the MTOS in the United States. Therefore, the present study’s findings, which were based on data accumulated from the time that the first trauma department in Korea began to operate, would be helpful in establishing the foundation needed to operate these recently developed regional trauma centers and trauma system.

The ISS was first suggested by Baker et al.[13] in 1974, and it is a method for classifying trauma patients based on anatomic measures. It was founded on the AIS, which was created with the goal of categorizing the severity and type of injuries caused by a car crash, and it is commonly used as an anatomical indicator to classify injury severity. The ISS is intuitive in severity evaluation and mortality prediction, and it is well associated with the mortality rate. However, it is accurately calculated only in the presence of objective data such as accurate test findings, surgery or autopsy results, etc.; thus, it has a limitation in assessing the severity of an emergency patient at an early stage.[14] Additionally, the ISS is limited in that severity is not accurately reflected in some trauma cases, as it does not indicate more than two overlapping injuries in the same area. Thus, some researchers have suggested the New Injury Severity Score (NISS) that sums the three highest AIS scores, regardless of the physical area.[15] However, like the ISS, it does not include physiological indices, making it difficult to accurately predict the prognosis. A few studies have reported that the NISS did not differ from or was even worse than the ISS for predicting mortality, admission to the ICU, and length of hospital stay,[16,17] and it is not widely used. In the present study, we confirmed the usefulness of the ISS in the Korean trauma patient population, and we found that its predictive ability was not poor. Particularly, the cut-off value of 15 confirmed the conventional criterion of using 15 points to define a severe trauma patient.[18]

The RTS was introduced in the early 1980s, and it is the most widely used physiologic scoring system to classify trauma patients.[17] It uses three physiological parameters, the GCS, SBP, and respiratory rate (RR), to score injuries, with a minimum score of 0 and a maximum score of 7.8408. A lower score indicates a more severe injury; thus, a lower survival rate is estimated. The RTS is excellent for predicting the mortality rate, it is suitable to use before the trauma patient is transported to a hospital because it is based on indices easily obtained in the field, and it is widely used. However, the GCS and RR cannot be accurately assessed in intubated or sedated patients. Moreover, in many developing countries or areas where a trauma system is not well established, prehospital records are incomplete or difficult to rely on, and many patients go to their local hospital first before they are transferred to a trauma center; hence, there is a limitation in obtaining physiological parameters early. Yet, some studies have reported significant results from routinely using the data measured immediately after patients are transferred to the hospital to score the injury without adjusting for the time passed after the injury.[14,19-22] We made great efforts to obtain ambulance service records and the medical records from transferring hospitals and reviewed them meticulously to determine the earliest physiological values; the cases for which we were not able to obtain accurate values were excluded. We believe that such efforts resulted in an AUC value comparable to that in previous research.[23]

The TRISS is a combined scoring system that was developed in 1987,[24] and it is based on the ISS as an anatomic component, RTS as a physiological component, and patient age as a comorbid component, resulting in a combined index for injury that complements each component. This scoring system is the most representative method to compute the probability of survival of a trauma patient, and it retrospectively applies a logistic regression model to compute the probability using the following equation; the solution values of which range from 0-1.

Probability of Survival = 1 ÷ (1 + e^-b)

b = α + β_AGE × AGE + β_RR × RR + β_SBP × SBP + β_GCS × GCS + β_ISS × ISS

In the equation, α, β_AGE, β_RR, β_SBP, β_GCS, and βISS are constants defined differently for patients with blunt trauma and those with penetrating trauma. In our study, we used the MTOS-derived coefficients revised in 1995.[25]

We computed the ROC curves of the ISS, RTS, and TRISS; estimated the respective AUCs; and statistically compared them to determine the ability of each scoring system to predict the mortality rate, which is the most important outcome measure of trauma. Generally, when an AUC value from ROC analysis is 0.9 or higher, the method is considered highly accurate.[26] In our study, the AUC of the TRISS was 0.942, suggesting that it is a very accurate method for predicting the mortality rate, and it was statistically better compared to the ISS and RTS. It has been consistently reported that the TRISS has superior results for predicting the mortality rate of trauma patients in developed countries compared to other scoring systems.[27-29] Thus, the TRISS may play a very important role in predicting mortality in a developing country such as Korea where the trauma patient population is different from that of a developed country and no trauma system has yet been established.

We believe that the respective cut-off value of the ISSSs is significant. In particular, the cut-off value of 15 for the ISS is consistent with the convention of classifying a severe patient with a high risk of death with a score ≥ 15. If the medical staff that specializes in trauma correctly use the AIS codes and apply the ISS according to its principles, the ISS can be a very useful tool for assessing the trauma patient population in Korea. The cut-off value for the TRISS was 0.9082 (90.82% survival probability). To effectively utilize these values as a tool for assessing the objective probability of death, both the positive predictive value and negative predictive value must be high; however, in the present study, the positive predictive value was 46.8% and the negative predictive value was 98.3%. Cayten et al.[27] reported that the positive predictive value of the TRISS for predicting death was 21.5%, and it was only 16% in patients who experienced a fall from standing height. Karmy-Jones et al.[30] reported that the positive predictive value of the TRISS was only 16.1%. Given these previous findings, our finding of 46.8% is comparable.

There are a few limitations in the study. First, the number of patients transferred from another hospital was as high as 3,400, which was almost half of the total sample. This may raise an issue regarding the measurement of the physiological parameters, which in turn affects the reliability of the RTS and TRISS results. To rectify this problem, we reviewed the prehospital records and consequently obtained results comparable to previously reported findings. Second, although the TRISS is a population-based scoring system, we applied constants that were computed based on the patient population of another region comparable to the Korean patients, and the study was conducted at a single trauma center. Recently, an epidemiological study conducted by a trauma center in Japan reported a significant difference between the coefficients computed from the Japanese data and the MTOS coefficients, and the authors argued that patient outcome in a certain area should be assessed using the TRISS coefficients, which are modified according to the area.[31] Considering this Japanese study, the present study’s findings cannot be generalized to the Korean trauma population. Therefore, new TRISS coefficients should be computed in a large-scale multi-center study with Korean trauma patients who have unique characteristics, or an outcome prediction model for Korean trauma patients should be developed.

When establishing a trauma system, a trauma scoring system is very important. To create a scoring system for the patient population in a developed country that can be used worldwide, the scoring system first has to be evaluated in a developing country that is different from a developed country in terms of its socioeconomic state and maturity of its trauma system. Thus, we used existing representative trauma scoring systems, i.e., the ISS, RTS, and TRISS, and compared the estimated mortality rates of trauma patients in a developing country. The results showed that these ISSSs can be very useful in assessing trauma patients in Korea, which is a developing country in terms of its trauma system, although the patient population is different from that of the country where the scoring systems were originally developed. Specifically, among the three ISSSs, the TRISS may be the most effective. However, additional research is needed to develop coefficients specific to Korea considering that the TRISS is a population-based scoring system.

Article information Continued