Early detection of bloodstream infection in critically ill children using artificial intelligence

Hye-Ji Han; Kyunghoon Kim; June Dong Park

doi:10.4266/acc.2024.00752

Articles

Page Path: HOME > Acute Crit Care > Volume 39(4); 2024 > Article

Original Article Pediatrics Early detection of bloodstream infection in critically ill children using artificial intelligence: Hye-Ji Han¹, Kyunghoon Kim^1,2, June Dong Park^2,3; Acute and Critical Care 2024;39(4):611-620.
DOI: https://doi.org/10.4266/acc.2024.00752
Published online: November 22, 2024

¹Department of Pediatrics, Seoul National University Bundang Hospital, Seongnam, Korea

²Department of Pediatrics, Seoul National University College of Medicine, Seoul, Korea

³Department of Pediatrics, Seoul National University Hospital, Seoul, Korea

Corresponding author: June Dong Park Department of Pediatrics, Seoul National University Children’s Hospital, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea Tel: +82-2-2072-3359 Fax: +82-2-762-3359 E-mail: jdparkmd@snu.ac.kr

• Received: June 24, 2024 • Revised: September 13, 2024 • Accepted: October 22, 2024

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

407 Views
29 Download

prev next

Full Article

Download PDF

Abstract
INTRODUCTION
MATERIALS AND METHODS
RESULTS
DISCUSSION
KEY MESSAGES
NOTES
REFERENCES

Abstract

Background
Despite the high mortality associated with bloodstream infection (BSI), early detection of this condition is challenging in critical settings. The objective of this study was to create a machine learning tool for rapid recognition of BSI in critically ill children.
Methods
Data were extracted from a derivative cohort comprising patients who underwent at least one blood culture during hospitalization in the pediatric intensive care unit (PICU) of a tertiary hospital from January 2020 to June 2023 for model development. Data from another tertiary hospital were utilized for external validation. Variables selected for model development were age, white blood cell count with segmented neutrophil count, C-reactive protein, bilirubin, liver enzymes, glucose, body temperature, heart rate, and respiratory rate. Algorithms compared were extra trees, random forest, light gradient boosting, extreme gradient boosting, and CatBoost.
Results
We gathered 1,806 measurements and recorded 290 hospitalizations from 263 patients in the derivative cohort. Median age on admission was 43 months, with an interquartile range of 10–118.75 months, and a male predominance was observed (n=160, 55.2%). Candida albicans was the most prevalent pathogen, and median duration to confirm BSI was 3 days (range, 3–4). Patients with BSI experienced significantly higher in-hospital mortality and prolonged stays in the PICU than patients without BSI. Random forest classifier achieved the highest area under the receiver operating characteristic curve of 0.874 (0.762 for the validation set).
Conclusions
We developed a machine learning model that predicts BSI with acceptable performance. Further research is necessary to validate its effectiveness.
Key Words: bloodstream infection; machine learning; sepsis

INTRODUCTION

Bloodstream infection (BSI) is a predominant healthcare-acquired infection that is being seen more frequently in intensive care units (ICUs) and is a leading cause of preventable death [1]. Mortality rates due to BSI range from 1.43% to 24% depending on the characteristics of the centers or pathogens [2]. BSI is responsible for 25%–62.2% of cases of sepsis, a condition characterized by a lethal, dysregulated host response with heterogeneous pathophysiology [3,4]. Heterogeneity of pathophysiology and treatment effects in sepsis have recently been identified as significant obstacles to novel therapies [5]. A personalized approach to treat sepsis based on phenotyping and rapid detection of BSI as a main treatable component would facilitate application of precision medicine in sepsis management. Additionally, sepsis associated with BSI is correlated with high mortality [6]. Prompt antibiotic therapy, which can be conceptualized as targeted management of sepsis resulting from severe infections like BSI, is of utmost importance in controlling septic shock, though its role in non-shock sepsis remains under debate [7]. This underlines the need for a high index of suspicion to identify critically ill patients at risk of BSI who can benefit from timely intervention with antibiotics [8].

However, accurate detection of BSI in critically ill patients can be hindered by various factors, complicating clinician decisions regarding the initiation of antibiotic treatments. A retrospective analysis from a high-volume institution revealed that clinician decisions generally have low positive predictive value in predicting BSI [9]. Clinical manifestations of infection in critically ill children may be vague due to extracorporeal devices causing heat loss [10,11]. Despite the high mortality associated with BSI in immunocompromised individuals, their only symptom might be a relatively low-grade fever without any localizing signs [12]. Efforts to diagnose BSI at an early stage have frequently resulted in the excessive use of blood cultures and high contamination rates, particularly in pediatric intensive care settings [13,14]. Blood culture contamination also remains a challenge, as it contributes to unnecessary antibiotic use, leading to antibiotic-resistance and adverse events that can be prevented [15]. A comprehensive tool for predicting BSI with readily available parameters will aid clinicians in decisions not only to start broad-spectrum antibiotics, but also to perform blood cultures when findings are clinically ambiguous.

Despite advances in sepsis-specific biomarkers, there is a lack of research on BSI-specific biomarkers and indicators in pediatric patients, especially those in critical care. With advancements in artificial intelligence (AI), which have facilitated more personalized predictions of clinical outcomes, as well as the availability of large datasets from real-time ICU monitoring, there is a growing interest in applying machine learning in the field of critical care. The aim of this study was to develop a precise predictive model using machine learning to identify pediatric patients at high risk of BSI, facilitating timely management and reducing complications associated with blood culture contamination.

MATERIALS AND METHODS

This de-identified analysis received approval from the Institutional Review Boards of Seoul National University Hospital and Seoul National University Bundang Hospital (IRB No. H-2402-133-1514, B-2401-879-111), including a waiver of informed consent.

Study Setting and Data Collection

This retrospective, observational study was carried out on patients admitted to the pediatric ICU (PICU) of a tertiary hospital in Korea from January 2020 to June 2023 for model development, with data from another tertiary hospital used for external validation. Flowchart of patient selection in this study is provided in Figure 1. We identified patients who were younger than 18 years and who underwent at least one blood culture during hospitalization. Blood cultures were all regarded as separate, independent events. The following clinical data were collected from our institution’s data warehouse: age, sex, discharge outcomes, ICU stay duration, blood culture results, heart rate (HR), respiratory rate (RR), body temperature (BT), and laboratory results (white blood cell count [WBC], percentage of segmented neutrophil [SEG], platelet count [PLT], glucose, total bilirubin [BIL], aspartate transaminase [AST], alanine transaminase [ALT], and C-reactive protein [CRP]).

Following the hospital’s protocol, more than two sets of blood cultures are collected and cultivated in BACTEC Peds Plus/F and Lytic/10 Anaerobic/F bottles (Becton Dickinson). Blood cultures are incubated for 5 days in a BACTEC FX Blood Culture System (Becton Dickinson).

Data Preprocessing

Each set of laboratory and hemodynamic measurements and blood culture results was synchronized within a 1-hour window, and missing values were imputed using forward filling. Non-physiological vital signs (30<HR<300, 5<RR<100, and 15<BT<50) were excluded from the analyses. Blood cultures with common commensals were classified as negative based on the National Health Safety Network guidelines for BSI, taking into account the likelihood of contamination [16]. Data preprocessing was conducted using the R statistical package (R 4.3.1; R Foundation for Statistical Computing).

Model Development and Validation

The dataset was randomly allocated into a training set and a test set at an 8:2 ratio for model development. The Synthetic Minority Over-sampling Technique (SMOTE) was employed to deal with unbalanced data, given the lower prevalence of BSI cases [17]. A machine learning model to predict BSI was developed using the following algorithms: extra trees, random forest, light gradient boosting, extreme gradient boosting, and CatBoost. Hyperparameters were fine-tuned using a grid search with five-fold cross-validation. Features related to systemic inflammation were chosen as input variables based on previous research on scoring systems or biomarkers. Correlations between BSI and clinical variables were examined (Figure 2), and feature selection was performed using Recursive Feature Elimination with Cross-Validation [18]. Following the exclusion of PLT and AST due to strong multicollinearity, age, WBC, SEG, CRP, BIL, ALT, glucose, BT, HR, and RR were used as input variables for model construction; all of these variables were selected for the final model. External validation was performed using an independent dataset. Python version 3.10.12 (Python Software Foundation; https://www.python.org) and open libraries, including Pandas, Numpy, and Scikit-learn, were used for model development.

Statistical Analysis

Clinical characteristics of patients with and without BSI were compared using the chi-square test for categorical variables and the Mann-Whitney U-test for continuous variables. P-values less than 0.05 were deemed statistically significant. Categorical variables are reported as frequencies and proportions, and continuous variables are presented as medians with interquartile ranges. Multivariate regression analyses were also performed to evaluate the significance of differences between groups. Model performances were primarily assessed using the area under the receiver operating characteristic curve (AUROC). A receiver operating characteristic curve, along with a precision-recall curve and a confusion matrix, were generated for the highest-performing model. R software version 4.3.1 was utilized to analyze baseline characteristics (R Foundation for Statistical Computing).

RESULTS

Baseline Characteristics

From a derivative cohort of 263 patients, 290 hospital admissions were recorded, with BSI present in 43 (14.83%) based on positive blood cultures. The overall mortality rate was 16.9% (n=49). The number of hospitalizations with at least one culture event, exhibiting parameters attributable to the systemic inflammatory response system (SIRS) criteria, was 52 (17.93%). SIRS was defined according to the consensual criteria of the International Consensus Conference on Pediatric Sepsis [19]. The median age of the cohort was 43 months (10–118.75), and there was a male predominance (n=160, 55.2%). Candida albicans was the most prevalent pathogen (n=17, 13.18%), and the median time to confirm a BSI was 3 days (range, 3–4). The group with BSI was significantly older at the time of admission than those without BSI (84 months [range, 10–173] vs. 41 months [range, 10–110], P=0.027). A significant difference in sex was observed. In addition to the presence of SIRS, the duration of PICU stay was longer (29 [range, 13.5–61] vs. 14 [range, 8–28.5], P<0.001) and the in-hospital mortality was higher (adjusted odds ratio, 2.96 [range, 1.37–6.25]; P=0.005) in the BSI group (Table 1).

A total of 1,806 culture events was analyzed, comparing measurements in events with and without BSI (Table 2). Notably, RR was slightly lower in the BSI group (P=0.049). Events that met SIRS criteria were more frequent in the BSI group (32.56% vs. 21.29%, P=0.004). Disparities in laboratory findings were significant between the two groups: WBC (P=0.002), SEG (P=0.001), BIL (P<0.001), ALT (P=0.004), and CRP (P<0.001) were higher in the BSI group than the non-BSI group.

Outcomes

The random forest model displayed the highest accuracy, with an AUROC of 0.874 for the development dataset. Its weighted precision and recall were 0.92 and 0.93, respectively, with a positive predictive value of 0.56. This model employed the “gini” criterion and was configured with a maximum depth of 30 for each of its 200 decision trees. The confusion matrix, alongside the receiver operating characteristic curve and the precision-recall curve for the random forest model, are presented in Figure 3. Feature importance for the best-performing model was assessed using both the mean accuracy decrease and permutation importance methods (Figure 4). For a validation dataset comprising 79 culture events and 10 positive blood cultures, the AUROC was 0.762. All positive blood cultures in the validation set were erroneously classified as negative. The receiver operating curves for the employed machine learning models are depicted in Figure 5.

DISCUSSION

In this study, we developed a machine learning model that demonstrated satisfactory performance in predicting blood culture results from clinical parameters. The model incorporated 12 input variables, including age, three vital signs, and eight laboratory results. The most effective model utilized a random forest classifier based on nine variables and achieved a commendable AUROC of 0.874 for the test set and 0.762 for the validation dataset. Children with BSI exhibited a relatively lower RR, whereas their WBC, SEG, BIL, ALT, and CRP values were higher than those of children without BSI. These clinical variables associated with positive blood cultures were more likely to be classified as a SIRS response. The older median age in the BSI group could be a contributing factor to the observed lower RR in this group. In good agreement with previous studies, children with BSI experienced higher mortality and required a longer ICU stay [2,6]. Outcomes were closely linked to BSI, regardless of SIRS status, which was more common in the BSI group. These results suggest that clinically silent BSI warrants attention. C. albicans, currently highlighted to become most prevalent in intensive care unit of Korea, was the most frequently identified pathogen in the derivative cohort in this study, whereas Staphylococcus aureus was the most common pathogen in the validation cohort [20] (Table 3). Nevertheless, the excessive use of blood cultures, driven by an effort to avert BSI-related mortality, may cause an increased rate of blood culture contamination. The contamination rate in our derivative cohort was 1.77% (n=43/1,806). By treating the growth of contaminants as negative blood cultures, the model promises robust prediction capabilities for genuine BSI in real-world practice.

Retrieval of culture results spanned 3 days (range, 3–4), even in the study’s relatively well-resourced hospital. Given the relatively prolonged period of time required to obtain culture results, several parameters and scoring systems that show a strong correlation with dysregulated host responses and inflammation have been developed to aid in risk stratification and outcome prediction in sepsis [21-23]. However, delayed alterations in parameters exhibited limited efficacy in real-time bedside prediction, and the newly introduced parameters remain impractical in low-resource settings. Procalcitonin, a well-established parameter for sepsis, shows inconsistent predictive accuracy [24]. Moreover, current scoring systems, including the recently developed Pheonix criteria, offer valuable prognostic insights but do not specifically target infection, and their therapeutic implications are unclear [5,23,25,26].

Prior attempts at predictive modeling of BSI, especially bacteremia, focused predominantly on adult patients and emergency department settings [24,27-31]. Many of these studies sourced their data from a single center. Bhavani et al. [24] proposed a gradient boosting machine learning model to predict BSI in hospitalized adult patients. This model showed impressive accuracy for fungemia (AUROC of 0.88) but lower accuracy for bacteremia (AUROC of 0.78). A limited number of single-center observational studies has developed machine learning models to predict sepsis or serious bacterial infections in specific pediatric populations, such as cancer patients or febrile infants younger than 60 days [32-36]. To our knowledge, predicting BSI in critically ill children remains an unmet need.

Our study has several limitations. Our findings should be interpreted cautiously due to the absence of adjustments for confounding factors like comorbidities. Because SIRS criteria were not fully evaluated, all SIRS values should be considered in further validation studies comparing the performance of the developed model with existing scoring systems. Furthermore, the selection of input variables was constrained by the exclusion of a few meaningful biomarkers and laboratory findings due to their low application rates with large sampling intervals, which potentially limited model performance. Missing information about potentially confounding factors, such as underlying disease and application of antibiotics or indwelling catheters, could have had significant effects on the performance of the developed model. Additionally, the blood culture collection procedure was not standardized, leading to a bias inherent in retrospective analyses. Last, despite employing SMOTE to address data imbalance, the relatively small BSI cohort sample size resulted in a decreased positive predictive value for the validation cohort.

In summary, we developed a precise machine learning model for BSI prediction to allow a tailored approach to control sepsis. Input variables for our developed model are readily available in low-resource ICU settings. Future research should explore the application of our prediction tool in various clinical scenarios and to various specimen types. Validation through extensive datasets is essential to assess the mortality reduction potential of AI-supported early antibiotic intervention, alongside comparisons with clinician decisions. Further investigations on comorbidities, immune status, infection sources, and pathogens will enhance the model utility, providing an important step toward high-quality precision medicine in sepsis.

KEY MESSAGES

▪ Bloodstream infection (BSI), one of the most common healthcare-associated infections, is increasing in incidence and can lead to the lethal condition of severe sepsis.

▪ Early detection is challenging in critically ill children.

▪ In our study, we developed a machine learning model with acceptable performance for predicting BSI in children, which may allow timely intervention.

NOTES

CONFLICT OF INTEREST

No potential conflict of interest relevant to this article was reported.

FUNDING

This work was supported by grant no. 09-2024-0001 from the Seoul National University Bundang Hospital Research Fund and partially supported by a grant (Kim Ki Eon Fund) from the Foundation for the Seoul National University Children’s Hospital in 2019.

ACKNOWLEDGMENTS

None.

AUTHOR CONTRIBUTIONS

Conceptualization: HJH. Data curation: HJH. Formal analysis: HJH. Funding acquisition: all authors. Methodology: HJH. Project administration: all authors. Visualization: HJH. Writing – original draft: HJH. Writing – review & editing: HJH, KK. All authors read and agreed to the published version of the manuscript.

Figure 1.

Flowchart of patient selection for model development and validation.

Figure 2.

Correlation matrix of the derivative cohort. To reduce multicollinearity, variables with lower correlation with bloodstream infection were eliminated if the correlation coefficient between any two variables exceeded 0.5. PLT: platelet; WBC: white blood cell count; BT: body temperature; HR: heart rate; RR: respiratory rate; BIL: bilirubin; AST: aspartate transaminase; ALT: alanine transaminase; CRP: C-reactive protein; SEG: segmented neutrophil; BSI: bloodstream infection.

Figure 3.

Predictive performance of the random forest model. (A) A confusion matrix of the derivative and validation cohorts. (B) Receiver operating characteristic curve. (C) Precision-recall curve were generated to evaluate the best-performing model. AUROC: area under the receiver operating curve.

Figure 4.

Feature importance of the random forest model. Feature importance was analyzed to determine the contributions of each feature to the performance of the model. (A) Mean decrease in impurity and (B) mean permutation importance were calculated and are plotted as bar graphs. Age (Age_m), C-reactive protein (CRP), alanine transaminase (ALT), bilirubin (BIL), and respiratory rate (RR) were given higher importance. MDI: mean decrease in impurity; WBC: white blood cell count; BT: body temperature; HR: heart rate; SEG: segmented neutrophil.

Figure 5.

Receiver operating characteristic curves of the machine learning models. Receiver operating characteristic curves for (A) The extreme gradient boosting model, (B) the extra trees classifier model, (C) the CatBoost model, and (D) the light gradient boosting model. AUROC: area under the receiver operating curve.

Table 1.

Baseline characteristics of patients in the derivative cohort

Variable	Blood culture		P-value
Variable	Positive (n=43)	Negative (n=247)	P-value
Age at admission (mo)	84 (10–173)	41 (10–110)	0.027
Male	27 (61.79)	133 (53.85)	0.356
SIRS	17 (39.53)	35 (14.17)	<0.001
In-hospital mortality	15 (34.88)	34 (13.77)	0.005
Length of ICU stay (day)	29 (13.5–61)	14 (8–28.5)	<0.001

Values are presented as median (interquartile range) or number (%).

SIRS: systemic inflammatory response system; ICU: intensive care unit.

The comparison of categorical variables between groups with and without bloodstream infection was performed using the chi-square test, while continuous variables were compared using the Mann-Whitney U-test. The differences in in-hospital mortality and length of intensive care unit stay between the two groups were analyzed using multivariate regression analysis.

Table 2.

Clinical and laboratory parameters of the derivative cohort

Variable	Blood culture		P-value
Variable	Positive (n=129)	Negative (n=1,677)	P-value
Heart rate (beats/min)	118 (100–135)	114 (98–135)	0.358
Respiratory rate (beats/min)	23.0 (20.0–30.3)	25.5 (20.0–33.6)	0.049
Body temperature (℃)	36.9 (36.6–37.4)	37 (36.6–37.5)	0.109
Glucose (mg/dl)	108 (94–130)	108 (93–129)	0.985
White blood cell count (×10³/ul)	10.0 (6.2–15.3)	8.3 (4.7–12.6)	0.002
Platelet count (/ul)	125 (69–223)	153 (78–286)	0.090
Segmented neutrophil (%)	73.8 (60.7–87.5)	70.3 (54.1–82.7)	0.001
Bilirubin (mg/dl)	1.0 (0.7–1.7)	0.8 (0.4–1.3)	<0.001
AST (U/L)	38 (22–76)	35 (24–70)	0.579
ALT (U/L)	38 (22–63)	30 (17–51)	0.004
CRP (mg/dl)	5.7 (0.8–13.5)	2.6 (0.6–7.0)	<0.001
SIRS	42 (32.6)	357 (21.3)	0.004

Values are presented as median (interquartile range) or number (%).

AST: aspartate transaminase; ALT: alanine transaminase; CRP: C-reactive protein; SIRS: systemic inflammatory response system.

Comparisons of categorical variables between groups with and without bacteremia were carried out using the chi-square test, while continuous variables were compared utilizing the Mann-Whitney U-test.

Table 3.

Isolated pathogens in the bloodstream infection of the cohorts

Pathogen	No. (%)
Derivative cohort (n=129)
Organisms
Candida albicans	17 (13)
Staphylococcus aureus	15 (12)
Escherichia coli	11 (9)
Klebsiella aerogenes	11 (9)
Acinetobacter baumannii	10 (8)
Klebsiella pneumoniae	8 (6)
Candida krusei	8 (6)
Staphylococcus epidermidis	8 (6)
Enterococcus faecalis	7 (5)
Pseudomonas aeruginosa	4 (3)
Bacillus species (non-B. cereus)	4 (3)
Candida parapsilosis	4 (3)
Serratia marcescens	3 (2)
Candida guilliermondii	2 (2)
Streptococcus species, viridans	2 (2)
Enterococcus faecium	2 (2)
Other	13 (10)
Validation cohort (n=10)
Staphylococcus aureus	7 (7)
Staphylococcus, Coagulase negative	1 (1)
Candida albicans	1 (1)
Klebsiella pneumoniae	1 (1)

REFERENCES

1. Hugonnet S, Harbarth S, Ferrière K, Ricou B, Suter P, Pittet D. Bacteremic sepsis in intensive care: temporal trends in incidence, organ dysfunction, and prognosis. Crit Care Med 2003;31:390-4.Article PubMed
2. Karagiannidou S, Triantafyllou C, Zaoutis TE, Papaevangelou V, Maniadakis N, Kourlaba G. Length of stay, cost, and mortality of healthcare-acquired bloodstream infections in children and neonates: a systematic review and meta-analysis. Infect Control Hosp Epidemiol 2020;41:342-54.Article PubMed
3. Huerta LE, Rice TW. Pathologic difference between sepsis and bloodstream infections. J Appl Lab Med 2019;3:654-63.Article PubMed PDF
4. Dolin HH, Papadimos TJ, Chen X, Pan ZK. Characterization of pathogenic sepsis etiologies and patient profiles: a novel approach to triage and treatment. Microbiol Insights 2019;12:1178636118825081. Article PubMed PMC PDF
5. Shah FA, Meyer NJ, Angus DC, Awdish R, Azoulay É, Calfee CS, et al. A research agenda for precision medicine in sepsis and acute respiratory distress syndrome: an official American Thoracic Society Research Statement. Am J Respir Crit Care Med 2021;204:891-901.Article PubMed PMC
6. Mellhammar L, Kahn F, Whitlow C, Kander T, Christensson B, Linder A. Bacteremic sepsis leads to higher mortality when adjusting for confounders with propensity score matching. Sci Rep 2021;11:6972. Article PubMed PMC PDF
7. Klompas M, Martin GS. Beyond septic shock: who else requires immediate antibiotics? Am J Respir Crit Care Med 2024;209:781-2.Article PubMed PMC
8. Morquin D, Lejeune J, Agostini C, Godreuil S, Reynes J, Le Moing V, et al. Time is of the essence: achieving prompt and effective antimicrobial therapy of bloodstream infection with advanced hospital information systems. Clin Infect Dis 2024;78:1434-42.Article PubMed PDF
9. Hoops KE, Fackler JC, King A, Colantuoni E, Milstone AM, Woods-Hill C. How good is our diagnostic intuition?: clinician prediction of bacteremia in critically ill children. BMC Med Inform Decis Mak 2020;20:144. Article PubMed PMC PDF
10. John JC, Taha S, Bunchman TE. Basics of continuous renal replacement therapy in pediatrics. Kidney Res Clin Pract 2019;38:455-61.Article
11. Lequier L, Horton SB, McMullan DM, Bartlett RH. Extracorporeal membrane oxygenation circuitry. Pediatr Crit Care Med 2013;14(5 Suppl 1):S7-12.Article PubMed PMC
12. Dropulic LK, Lederman HM. Overview of infections in the immunocompromised host. Microbiol Spectr 2016;4:DMIH2-0026-2016. Article PDF
13. Sacchetti B, Travis J, Steed LL, Webb G. Identification of the main contributors to blood culture contamination at a tertiary care academic medical center. Infect Prev Pract 2022;4:100219. Article
14. Ramirez P, Gordón M, Cortes C, Villarreal E, Perez-Belles C, Robles C, et al. Blood culture contamination rate in an intensive care setting: effectiveness of an education-based intervention. Am J Infect Control 2015;43:844-7.Article PubMed
15. Schinkel M, Boerman A, Carroll K, Cosgrove SE, Hsu YJ, Klein E, et al. Impact of blood culture contamination on antibiotic use, resource utilization, and clinical outcomes: a retrospective cohort study in Dutch and US hospitals. Open Forum Infect Dis 2023;11:ofad644. Article PubMed PMC PDF
16. Centers for Disease Control and Prevention. Bloodstream infection event (central line-associated bloodstream infection and non-central line-associated bloodstream infection). Device-associated Module BSI. Centers for Disease Control and Prevention; 2017.
17. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 2002;16:321-57.Article PDF
18. Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Mach Learn 2002;46:389-422.Article
19. Goldstein B, Giroir B, Randolph A, International Consensus Conference on Pediatric Sepsis. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med 2005;6:2-8.Article PubMed
20. Kim EJ, Lee E, Kwak YG, Yoo HM, Choi JY, Kim SR, et al. Trends in the epidemiology of candidemia in intensive care units from 2006 to 2017: results from the Korean National Healthcare-Associated Infections Surveillance System. Front Med (Lausanne) 2020;7:606976. Article PubMed PMC
21. Ishaque S, Famularo ST 3rd, Saleem AF, Siddiqui NU, Kazi Z, Parkar S, et al. Biomarker-based risk stratification in pediatric sepsis from a low-middle income country. Pediatr Crit Care Med 2023;24:563-73.Article PubMed PMC
22. McLymont N, Glover GW. Scoring systems for the characterization of sepsis and associated outcomes. Ann Transl Med 2016;4:527. Article PubMed PMC
23. Sanchez-Pinto LN, Bennett TD, DeWitt PE, Russell S, Rebull MN, Martin B, et al. Development and validation of the phoenix criteria for pediatric sepsis and septic shock. JAMA 2024;331:675-86.Article PubMed PMC
24. Bhavani SV, Lonjers Z, Carey KA, Afshar M, Gilbert ER, Shah NS, et al. The development and validation of a machine learning model to predict bacteremia and fungemia in hospitalized patients using electronic health record data. Crit Care Med 2020;48:e1020-8.Article PubMed PMC
25. Harbarth S, Holeckova K, Froidevaux C, Pittet D, Ricou B, Grau GE, et al. Diagnostic value of procalcitonin, interleukin-6, and interleukin-8 in critically ill patients admitted with suspected sepsis. Am J Respir Crit Care Med 2001;164:396-402.Article PubMed
26. Póvoa P, Coelho L, Dal-Pizzol F, Ferrer R, Huttner A, Conway Morris A, et al. How to use biomarkers of infection or sepsis at the bedside: guide to clinicians. Intensive Care Med 2023;49:142-53.Article PubMed PMC PDF
27. Choi DH, Hong KJ, Park JH, Shin SD, Ro YS, Song KJ, et al. Prediction of bacteremia at the emergency department during triage and disposition stages using machine learning models. Am J Emerg Med 2022;53:86-93.Article PubMed
28. Tsai WC, Liu CF, Ma YS, Chen CJ, Lin HJ, Hsu CC, et al. Real-time artificial intelligence system for bacteremia prediction in adult febrile emergency department patients. Int J Med Inform 2023;178:105176. Article PubMed
29. Lee KH, Dong JJ, Kim S, Kim D, Hyun JH, Chae MH, et al. Prediction of bacteremia based on 12-year medical data using a machine learning approach: effect of medical data by extraction time. Diagnostics (Basel) 2022;12:102. Article PubMed PMC
30. Zhang F, Wang H, Liu L, Su T, Ji B. Machine learning model for the prediction of gram-positive and gram-negative bacterial bloodstream infection based on routine laboratory parameters. BMC Infect Dis 2023;23:675. Article PubMed PMC PDF
31. Goh V, Chou YJ, Lee CC, Ma MC, Wang WY, Lin CH, et al. Predicting bacteremia among septic patients based on ED information by machine learning methods: a comparative study. Diagnostics (Basel) 2022;12:2498. Article PubMed PMC
32. Alali M, Mayampurath A, Dai Y, Bartlett AH. A prediction model for bacteremia and transfer to intensive care in pediatric and adolescent cancer patients with febrile neutropenia. Sci Rep 2022;12:7429. Article PubMed PMC PDF
33. Sung L, Corbin C, Steinberg E, Vettese E, Campigotto A, Lecce L, et al. Development and utility assessment of a machine learning bloodstream infection classifier in pediatric patients receiving cancer treatments. BMC Cancer 2020;20:1103. Article PubMed PMC PDF
34. Tsai CM, Lin CR, Zhang H, Chiu IM, Cheng CY, Yu HR, et al. Using machine learning to predict bacteremia in febrile children presented to the emergency department. Diagnostics (Basel) 2020;10:307. Article PubMed PMC
35. Yang Y, Wang YM, Lin CR, Cheng CY, Tsai CM, Huang YH, et al. Explainable deep learning model to predict invasive bacterial infection in febrile young infants: a retrospective study. Int J Med Inform 2023;172:105007. Article PubMed
36. Brouwer L, Cunney R, Drew RJ. Predicting community acquired bloodstream infection in infants using full blood count parameters and C-reactive protein; a machine learning study. Eur J Pediatr 2024;183:2983-93.Article PubMed PMC PDF

Figure & Data

References

Citations

Citations to this article as recorded by

PubReader
ePub Link

Cite

CITE: export Copy Format; Close

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

RIS — For EndNote, ProCite, RefWorks, and most other reference management software
BibTeX — For JabRef, BibDesk, and other BibTeX-specific software

Include:

Citation for the content below
Citation and abstract for the content below

Early detection of bloodstream infection in critically ill children using artificial intelligence

Acute Crit Care. 2024;39(4):611-620. Published online November 22, 2024

DOI: https://doi.org/10.4266/acc.2024.00752

XML Download

Figure

Early detection of bloodstream infection in critically ill children using artificial intelligence

Figure 1. Flowchart of patient selection for model development and validation.

Figure 2. Correlation matrix of the derivative cohort. To reduce multicollinearity, variables with lower correlation with bloodstream infection were eliminated if the correlation coefficient between any two variables exceeded 0.5. PLT: platelet; WBC: white blood cell count; BT: body temperature; HR: heart rate; RR: respiratory rate; BIL: bilirubin; AST: aspartate transaminase; ALT: alanine transaminase; CRP: C-reactive protein; SEG: segmented neutrophil; BSI: bloodstream infection.

Figure 3. Predictive performance of the random forest model. (A) A confusion matrix of the derivative and validation cohorts. (B) Receiver operating characteristic curve. (C) Precision-recall curve were generated to evaluate the best-performing model. AUROC: area under the receiver operating curve.

Figure 4. Feature importance of the random forest model. Feature importance was analyzed to determine the contributions of each feature to the performance of the model. (A) Mean decrease in impurity and (B) mean permutation importance were calculated and are plotted as bar graphs. Age (Age_m), C-reactive protein (CRP), alanine transaminase (ALT), bilirubin (BIL), and respiratory rate (RR) were given higher importance. MDI: mean decrease in impurity; WBC: white blood cell count; BT: body temperature; HR: heart rate; SEG: segmented neutrophil.

Figure 5. Receiver operating characteristic curves of the machine learning models. Receiver operating characteristic curves for (A) The extreme gradient boosting model, (B) the extra trees classifier model, (C) the CatBoost model, and (D) the light gradient boosting model. AUROC: area under the receiver operating curve.

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Early detection of bloodstream infection in critically ill children using artificial intelligence

Variable	Blood culture		P-value
Variable	Positive (n=43)	Negative (n=247)	P-value
Age at admission (mo)	84 (10–173)	41 (10–110)	0.027
Male	27 (61.79)	133 (53.85)	0.356
SIRS	17 (39.53)	35 (14.17)	<0.001
In-hospital mortality	15 (34.88)	34 (13.77)	0.005
Length of ICU stay (day)	29 (13.5–61)	14 (8–28.5)	<0.001

Variable	Blood culture		P-value
Variable	Positive (n=129)	Negative (n=1,677)	P-value
Heart rate (beats/min)	118 (100–135)	114 (98–135)	0.358
Respiratory rate (beats/min)	23.0 (20.0–30.3)	25.5 (20.0–33.6)	0.049
Body temperature (℃)	36.9 (36.6–37.4)	37 (36.6–37.5)	0.109
Glucose (mg/dl)	108 (94–130)	108 (93–129)	0.985
White blood cell count (×10³/ul)	10.0 (6.2–15.3)	8.3 (4.7–12.6)	0.002
Platelet count (/ul)	125 (69–223)	153 (78–286)	0.090
Segmented neutrophil (%)	73.8 (60.7–87.5)	70.3 (54.1–82.7)	0.001
Bilirubin (mg/dl)	1.0 (0.7–1.7)	0.8 (0.4–1.3)	<0.001
AST (U/L)	38 (22–76)	35 (24–70)	0.579
ALT (U/L)	38 (22–63)	30 (17–51)	0.004
CRP (mg/dl)	5.7 (0.8–13.5)	2.6 (0.6–7.0)	<0.001
SIRS	42 (32.6)	357 (21.3)	0.004

Pathogen	No. (%)
Derivative cohort (n=129)
Organisms
Candida albicans	17 (13)
Staphylococcus aureus	15 (12)
Escherichia coli	11 (9)
Klebsiella aerogenes	11 (9)
Acinetobacter baumannii	10 (8)
Klebsiella pneumoniae	8 (6)
Candida krusei	8 (6)
Staphylococcus epidermidis	8 (6)
Enterococcus faecalis	7 (5)
Pseudomonas aeruginosa	4 (3)
Bacillus species (non-B. cereus)	4 (3)
Candida parapsilosis	4 (3)
Serratia marcescens	3 (2)
Candida guilliermondii	2 (2)
Streptococcus species, viridans	2 (2)
Enterococcus faecium	2 (2)
Other	13 (10)
Validation cohort (n=10)
Staphylococcus aureus	7 (7)
Staphylococcus, Coagulase negative	1 (1)
Candida albicans	1 (1)
Klebsiella pneumoniae	1 (1)

Table 1. Baseline characteristics of patients in the derivative cohort

Values are presented as median (interquartile range) or number (%).

SIRS: systemic inflammatory response system; ICU: intensive care unit.

Table 2. Clinical and laboratory parameters of the derivative cohort

Values are presented as median (interquartile range) or number (%).

AST: aspartate transaminase; ALT: alanine transaminase; CRP: C-reactive protein; SIRS: systemic inflammatory response system.

Comparisons of categorical variables between groups with and without bacteremia were carried out using the chi-square test, while continuous variables were compared utilizing the Mann-Whitney U-test.

Table 3. Isolated pathogens in the bloodstream infection of the cohorts