Predicting open wound mortality in the ICU using machine learning

Introduction

Open wounds in the skin have a significant impact on patients’ health (1). Labeled as a comorbid condition, open wounds represent a “silent epidemic” that affect a large portion of the United States population. Although most acute wounds heal without issue, unhealed wounds cost about $25 billion a year in management (2). Chronic wounds are very rarely seen in healthy patients; their care represents a cross section of many medical disciplines, diabetes, trauma, hypertension, vascular insufficiency, and rheumatologic diseases. While diabetes and obesity are well-known mortality risk factors, these diagnoses and similar ones tend to minimize the true role chronic wounds play in a patient’s morbidity (3). A patient in the ICU with a chronic wound of diabetic etiology and another post-surgical wound from a pressure ulcer are not considered a sum, rather as individual complications of diabetes and post-op immobility. Many of these problems result in increased ICU stays and mortality (4).

Clinical and risk prediction models can help clinicians make better treatment decisions (5). In the literature, two well-established algorithms predict the mortality risk in the general population: the Charlson Comorbidity Index (CCI) and the Elixhauser Comorbidity Index (6). Both models use ICD-9-CM diagnosis codes to stratify patients. The two algorithms rely on symptom severity and have been validated in multiple studies. They have also been applied to ICU patients yet they have not been validated to patients with chronic wounds. Machine learning algorithms have been applied to the ICU settings but never specifically to the chronic wound patient population in the ICU (7,8). Established ICU models such as the Acute Physiology and Chronic Health Evaluation (APACHE IV) and Sequential Organ Failure Assessment (SOFA) require many variables and about 37.3 minutes on average to enter the necessary input data to get a result (9). In the ICU setting, algorithms with the least number of variables are of high clinical utility (10).

In this context, patients with open wounds may be facing an increased risk of ICU stay and mortality. There is a dearth of studies that investigate mortality among patients with open wounds in the ICU. In this study, we sought to develop a model that predicts the risk of mortality among those patients in the ICU. We present the following article in accordance with the Materials Design Analysis Reporting (MDAR) reporting checklist (available at http://dx.doi.org/10.21037/jeccm-20-154).

Methods

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Patient data from the Medical Information Mart for Intensive Care III (MIMIC-III) database were used for this study (11). MIMIC-III is a publicly available database that includes deidentified electronic health record (EHR) data for patients who stayed in critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012 and signed informed consent. This study did not require institutional review board approval as it used publicly available data. Patients with open wound diagnoses were selected using the ICD-9-CM codes 870.0 to 906.0.

Statistical analysis

The Julia programming language (v0.5) was used for data analyses at our institution’s Unified Research data Sharing and Analysis (URSA) Stronghold. Data processing was facilitated with MySQL and Julia packages including DataFrames.jl (to create data tables) and PredictMD.jl (for prediction modeling).

For each patient selected, CCI and Elixhauser scores were calculated. CCI scores range from 0 to 34 and incorporated comorbidities in 17 categories with more points assigned to severe comorbidities as reported in its guidelines [2 (hemiplegia, moderate or severe renal disease, diabetes with end stage-organ damage, tumor without metastasis, leukemia, lymphoma); 3 (moderate or severe liver disease); 6 (metastatic solid tumor, AIDS)]. Elixhauser scores range from 0 to 36 and were scored by assigning one point per comorbidity for a total of 36 categories (12).

Mortality during ICU stay was our primary outcome of interest. A new variable was created with value of either “0” (indicating that the patient did not die during ICU stay) or “1” (indicating that the patient died during the ICU stay). Model creation and fitting was conducted using the PredictMD.jl package. The random forest model developed to predict the mortality used six variables: wound location, gender, age, admission type, minimum platelet count, and hyperphosphatemia. The predictors were selected after careful review of the literature (13-16). Wound location was defined by ICD-9-CM codes designating the trunk, the extremities or the head and neck. Admission type included “elective, emergency, or urgent.” Minimum platelet count and hyperphosphatemia (>4.5 mg/dL) were calculated for each patient using lab data. Patients with a mortality occurring within 24 hours of admission were excluded. The data were randomly partitioned into a training set (70%) and testing set (30%). The training set was used to develop and train a random forest model for predicting mortality, with the predictive features as inputs and the mortality outcome as output. The testing set was then used to compute the sensitivity and specificity of the random forest model at different operating thresholds. A receiver operating characteristic (ROC) curve was generated with a resulting area under the curve (AUC) value. The area under the ROC curve was calculated using trapezoidal integration. These analyses were performed in the Julia programming language using the PredictMD toolkit.

Results

A total of 3,937 patients out of 61,532 ICU admissions were included in the study; 38.4% were female and 61.6% were male. Admission type was urgent (1.9%), elective (6.0%) or emergent (92.1%). The mean age was 76.57. Of those, 3,372 (85%) survived and 565 (15%) died during their ICU stay. Different types of insurance were represented in the study population. Most were on Medicare (56.7%), or private insurance (30.8%). Table 1 summarizes the patient demographics and statistics.

Table 1 Summary statistics on the cohort of patients who died and those who survived
Full table

The data were randomly split into a training set of 2,756 patients and a testing set of 1,181 patients. The training set was used to develop and train a random forest machine learning model for predicting mortality. The testing set was used to evaluate the performance of the random forest model and generate a ROC curve (Figure 1). The model achieved an AUC of 0.924. In comparison, the CCI and Elixhauser models resulted in AUC scores of 0.528 and 0.565 (Figure 1). The random forest model was found to have a sensitivity of 98% and a specificity of 85% with a recall of 94%, 79% precision and 0.73 Cohen’s Kappa statistic. Figure 2 shows the classifier score histogram for the model. P<0.05 was chosen as the level of significance. The two variables with the highest relative contribution in predicting mortality were minimum platelet count and hyperphosphatemia (Table 2).

Figure 1 Receiver operating curve of (A) the random forest model; (B) the Charlson Comorbidity Index (CCI) model; (C) the Elixhauser model.

Figure 2 Classifier score histogram.

Table 2 Random forest modeling results with coefficients and statistical significance
Full table

Discussion

Managing open wound patients in the ICU presents a unique challenge since their condition may have an impact on survival. Our data suggest that using only six readily available variables, clinicians may be able to predict with great accuracy which patients are at higher risk of dying in the ICU.

The CCI and the Elixhauser models compute risk scores using ICD-9-CM codes. CCI uses severity of symptoms in its calculation while Elixhauser uses comorbidity categories without severity. Both of these models have been applied to the ICU stetting but never specifically to the open wound patient population specifically. Our predictive model outperformed both CCI and Elixhauser with fewer variables, indicating its potentially high practical clinical utility.

Patients with ICD-9-CM codes 870.0 to 906.0 included in this study may not have had their open-wound(s) as the reason for their ICU admission. However, understanding which variables can contribute to mortality in patients with open wounds such as pressure ulcers in the ICU may be used to inform the development of targeted public health measures and potentially enhance discussions around wound morbidity (17). These wounds can lead to multiple complications including gangrene, hemorrhage and infection requiring lower-extremity amputations. The disabilities caused by those wounds worsen their healing resulting in a vicious cycle (18).

Even though our random forest model was developed using ICU patient data, further research could investigate its use in different settings. At the threshold of 0.13, our random forest model has a sensitivity of 98% and a specificity of 85%. With a high specificity and specificity, our model could be useful in prioritizing admission to the ICU by lowering the chances of false positives.

One potential limitation to this study is possible class imbalance affecting the model’s accuracy. At present, the random forest model presented here is unvalidated in a different hospital ICU. Future studies could validate the model with prospective cohorts, increasing the number of patients to minimize class imbalance. In this study, the model was compared to the CCI and the Elixhauser indexes since they are based on the ICD-9-CM diagnosis codes for each patient. The Elixhauser score estimates the comorbidity burden but not the acute fluctuation in the patient’s illness. Further comparison could test the random forest model against other commonly used algorithms such as APACHE-II and SOFA to estimate the model efficiency and time to completion when compared to the gold standard. The random forest model should also be assessed in racially and geographically diverse US populations to assess if the predictive model still outperforms the CCI and Elixhauser models. The goal of predictive models in the healthcare setting is to improve patient care. Machine learning should be used as adjunct and not as a substitute to clinical decision making.

Conclusions

Comorbidities continue be labeled as the leading factors of mortality in patients with open wound diagnoses. The random forest model developed in this study predicts the risk of mortality among those patients in the ICU. Our model suggests that using six readily available variables only, clinicians may be able to use predictive modeling to identify patients who are high risk for mortality in the ICU.

Acknowledgments

The authors thank Dilum Aluthge and Ishan Sinha for their assistance with PredictMD.

Funding: This work was funded in part by the Scholarly Concentration Program in the Warren Alpert Medical School of Brown University and National Institutes of Health grant U54GM115677.

Footnote

Reporting Checklist: The authors have completed the Materials Design Analysis Reporting (MDAR) reporting checklist. Available at http://dx.doi.org/10.21037/jeccm-20-154

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jeccm-20-154). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Jarbrink K, Ni G, Sonnergren H, et al. Prevalence and incidence of chronic wounds and related complications: a protocol for a systematic review. Syst Rev 2016;5:152. [Crossref] [PubMed]
2. Sen CK, Gordillo GM, Roy S, et al. Human skin wounds: a major and snowballing threat to public health and the economy. Wound Repair Regen 2009;17:763-71. [Crossref] [PubMed]
3. Bhupathiraju SN, Hu FB. Epidemiology of Obesity and Diabetes and Their Cardiovascular Complications. Circ Res 2016;118:1723-35. [Crossref] [PubMed]
4. Cullen Gill E. Reducing hospital acquired pressure ulcers in intensive care. BMJ Qual Improv Rep 2015;4:u205599.w3015.
5. Vogenberg FR. Predictive and prognostic models: implications for healthcare decision-making in a modern recession. Am Health Drug Benefits 2009;2:218-22. [PubMed]
6. Austin SR, Wong YN, Uzzo RG, et al. Why Summary Comorbidity Measures Such As the Charlson Comorbidity Index and Elixhauser Score Work. Med Care 2015;53:e65-72. [Crossref] [PubMed]
7. Rojas JC, Carey KA, Edelson DP, et al. Predicting Intensive Care Unit Readmission with Machine Learning Using Electronic Health Record Data. Ann Am Thorac Soc 2018;15:846-53. [Crossref] [PubMed]
8. Pollard TJ, Celi LA. Enabling Machine Learning in Critical Care. ICU Manag Pract 2017;17:198-9. [PubMed]
9. Kuzniewicz MW, Vasilevskis EE, Lane R, et al. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest 2008;133:1319-27. [Crossref] [PubMed]
10. Anand RS, Stey P, Jain S, et al. Predicting Mortality in Diabetic ICU Patients Using Machine Learning and Severity Indices. AMIA Jt Summits Transl Sci Proc 2018;2017:310-9. [PubMed]
11. Johnson AE, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016;3:160035 [Crossref] [PubMed]
12. Metcalfe D, Masters J, Delmestri A, et al. Coding algorithms for defining Charlson and Elixhauser co-morbidities in Read-coded databases. BMC Med Res Methodol 2019;19:115. [Crossref] [PubMed]
13. Szpaderska AM, Egozi EI, Gamelli RL, et al. The effect of thrombocytopenia on dermal wound healing. J Invest Dermatol 2003;120:1130-7. [Crossref] [PubMed]
14. Guo S, Dipietro LA. Factors affecting wound healing. J Dent Res 2010;89:219-29. [Crossref] [PubMed]
15. Maroz N, Simman R. Wound Healing in Patients With Impaired Kidney Function. J Am Coll Clin Wound Spec 2014;5:2-7. [Crossref] [PubMed]
16. Gerstein AD, Phillips TJ, Rogers GS, et al. Wound healing and aging. Dermatol Clin 1993;11:749-57. [Crossref] [PubMed]
17. Coyer F, Tayyib N. Risk factors for pressure injury development in critically ill patients in the intensive care unit: a systematic review protocol. Syst Rev 2017;6:58. [Crossref] [PubMed]
18. Bhattacharya S, Mishra RK. Pressure ulcers: Current understanding and newer modalities of treatment. Indian J Plast Surg 2015;48:4-16. [Crossref] [PubMed]