Severity of illness scores at presentation predict ICU admission and mortality in COVID-19

Introduction

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has led to over 7.5 million global cases and 425,000 deaths due to coronavirus disease 2019 (COVID-19) as of June 13, 2020. An emerging challenge in the COVID-19 pandemic is identifying, among the patients presenting to the hospital, which patients are likely to require intensive care unit (ICU) and which can be managed without such intensive resources. Predicting ICU admission would help allocate clinical resources and allow prognostic enrichment in clinical trials. Various severity of illness scores has been developed to predict the risk of mortality at the time of ICU admission, including the sequential organ failure assessment (SOFA) (1), simplified acute physiology score (SAPS) II (2), and acute physiology and chronic health evaluation (APACHE) II (3). These scores have also demonstrated utility in evaluating disease severity outside of the ICU setting. SOFA score at the time of emergency department presentation predicts outcomes in severe sepsis (4). The APACHE-II predicts mortality of acute pancreatitis at initial presentation (5) and the long-term mortality of patients admitted with a COPD exacerbation outside of the ICU (6). Higher APACHE-II and SAPS-II scores also correlated with subsequent ICU admission in patients admitted to a tertiary intermediate care unit (7). The APACHE-II score was recently shown to predict ICU mortality in COVID-19 (8), but its performance at the time of hospital presentation and its ability to predict ICU admission for patients with COVID-19 remain unknown. Further, the performance of these risk scores, relative to newly proposed COVID-specific markers of severity of illness such as the neutrophil:lymphocyte ratio, have yet to be evaluated (9).

The performance of the SOFA, SAPS-II, APACHE-II severity of illness scores and the neutrophil:lymphocyte ratio were evaluated for their ability to predict ICU in a retrospective cohort of patients hospitalized for COVID-19 at a large academic medical center. We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/jeccm-20-92).

Methods

Setting and participants

This retrospective cohort study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the Vanderbilt University institutional review board with wavier of informed consent (IRB#200537). Between March 8^th and May15^th, 2020, we enrolled all adult patients (18 years or older) who were admitted from the emergency department (ED) to Vanderbilt University Medical Center in Nashville, TN with symptomatic COVID-19, confirmed by SARS-CoV-2 testing. Patient outcomes were followed until June 12^th at 5pm. Patients readmitted within 72 hours of discharge were analyzed as a single admission.

Variables and data sources

Study personnel reviewed electronic health records to collect baseline patient characteristics including age, gender, race, ethnicity, body mass index, home medications, comorbidities, and smoking history. Active malignancy was defined as receipt of chemotherapy, radiotherapy, or surgery for malignancy within 45 days of admission. Vital signs and laboratory values were recorded. Missing data was analyzed by pairwise deletion. The primary outcome was ICU admission at any point during the inpatient hospitalization. Secondary outcomes were mortality and hospital length of stay. Patients who expired without ICU transfer based on limitations in care (e.g., a patient who expired after transfer from the hospital ward to inpatient hospice) were analyzed as not experiencing ICU transfer. Patients discharged to hospice were analyzed as having died. Patients who remained hospitalized at the end of the follow-up period were censored for analyses of mortality.

Measurement of severity of illness scores

The SOFA (1), SAPS-II (2), APACHE-II (3), and qSOFA (10), were calculated using data collected within 24 hours of ED presentation. Calculation of SOFA scores substituted the oxygen saturation to fraction of inspired oxygen (S/F) ratio (11) in place of the PaO₂ to fraction of inspired oxygen (P/F) ratio. APACHE-II scores were calculated without inclusion of the PaO₂ term (12) and using bicarbonate instead of pH (13). Scores were calculated blinded to patient outcome.

Statistical Analysis

Sample size was estimated by the precision analysis. With the proposed sample size of 110, and an estimated ICU admission rate of 30%, the half-width of the two-sided 95% confidence interval (CI) for receiver operating characteristic (ROC) curve’s area under the curve (AUC) was less than 12%. Continuous variables were reported as median and interquartile range (IQR). Categorical variables were reported as frequencies and proportions. Between-group comparisons were made with the Mann-Whitney rank-sum test for continuous variables and the Fishers exact test for categorical variables. Comparisons between AUCs were made using Delong’s test. A two-sided P value <0.05 was used to indicate statistical significance without adjustment for multiple testing. ROC curves with 95% CI: for qSOFA score, SOFA score, SAPS-II score, APACHE-II score, and neutrophil:lymphocyte ratio were generated using R version 4.0 (14) and package pROC 1.16.2 (15).

Results

Patient cohort

A patient flow diagram is shown in Figure 1. One hundred twenty-eight patients with laboratory confirmed SARS-CoV-2 were admitted from the ED during the study period from March 8, 2020 to May 15, 2020. At the end of the follow-up period on June 12, 2020, 125 patients were discharged and three patients remained hospitalized, all of whom had already required ICU admission.

Figure 1 Patient flow diagram. 128 patients were admitted from an emergency department (ED) with COVID-19. At the end of follow-up, 86 patients recovered without requiring intensive care unit (ICU) care, 25 patients recovered after requiring ICU admission, 3 patients died after declining ICU care, 11 patients died after receiving ICU care, and 3 patients remained hospitalized.

ICU admissions

Overall, 39/128 (30.4%) of patients received ICU care. Table 1 shows baseline patient characteristics for patients who did and did not require ICU admission. The most common indication for ICU admission was hypoxemic respiratory failure (n=32, 82.1%). Other indications included altered mental status (n=2, 5.1%), arrhythmia (n=1, 2.6%) and increased nursing needs (n=4, 10.3%). Twenty-four patients (61.5%) were admitted directly to the ICU, while 15 (38.4%) patients were initially admitted to the ward and subsequently transferred to the ICU. The median time from presentation to ICU transfer was 3.0 days with IQR from 1.0 to 5.5 days.

Table 1 Clinical characteristics
Full table

In univariate analysis, patients who required ICU admission were more likely to have diabetes mellitus (38.5% vs. 19.1%, P=0.03), active malignancy (10.3% vs. 1.1%, P=0.03), or require supplemental oxygen at admission (79.5% vs. 40.4%, P<0.001). Gender and race were not significantly associated with risk of ICU admission.

Figure 2A shows the ROC curves for the three severity of illness scores. The C-index for the APACHE-II score (AUC =0.851, 95% CI: 0.786 to 0.917) was higher than the SAPS-II (AUC =0.758, 95% CI: 0.671 to 0.844, P=0.009), SOFA score (AUC =0.730, 95% CI: 0.642 to 0.817, P=0.003), and qSOFA score (AUC 0.713, 95% CI: 0.630 to 0.797, P=0.004). The neutrophil:lymphocyte ratio alone had a C-index of 0.756, but the addition of the neutrophil:lymphocyte ratio to the APACHE-II did not significantly improve predictive ability of the APACHE-II score (P=0.445). No patient with an APACHE-II score <8 experienced ICU admission during hospitalization, and 18.2% of patients with an APACHE-II score between 8 and 10 experienced ICU admission. Similarly, no patient with qSOFA score 0 experienced ICU admission during hospitalization, and 26.4% of patients with qSOFA score 1 experienced ICU admission. All patients who did not require supplemental oxygen on admission but still experienced critical illness had APACHE-II scores greater than 10. Figure 2B,C display the risk of ICU admission stratified by APACHE-II and qSOFA score, respectively. More patients had an APACHE-II score <8 than qSOFA score of 0 (35/128 vs. 19/128, P=0.21).

Figure 2 Severity of illness scores and intensive care unit (ICU) admission. For patients with COVID-19 presenting to the emergency department (ED), (A) displays the receiver operating characteristic curves for the outcome of ICU admission for the acute physiology and chronic health evaluation (APACHE) II score (AUC 0.851), simplified acute physiology score (SAPS) II (AUC 0.758), sequential organ failure assessment (SOFA) score (AUC 0.730), and quick SOFA (qSOFA) score (AUC 0.713); (B) displays the percent of patients who experienced ICU admission by APACHE-II score at presentation to the ED; (C) displays the percent of patients who experienced ICU admission by qSOFA score at presentation to the ED.

Patient outcomes

111 patients survived to hospital discharge, and 14 patients died. Three critically ill patients remained hospitalized at the end of follow-up. At the end of follow-up, the overall mortality was 10.9% for all patients and 30.5% for ICU patients.

All three models predicted in-hospital mortality. The SAPS-II score had the highest AUC (0.911, 95% CI: 0.856 to 0.966). The SAPS-II score did not perform better than either the APACHE-II (AUC =0.851, 95% CI: 0.766 to 0.936, P=0.072) or SOFA (AUC =0.826, 95% CI: 0.732 to 0.919, P=0.068), but did have a better AUC than the qSOFA (AUC =0.801, 95% CI: 0.692 to 0.911, P=0.028). The ROC curve of all three severity of illness scores is shown in Figure 3A. Figure 3B,C display the mortality risk stratified by APACHE-II and qSOFA score, respectively. No patient with an APACHE-II score less than 10 or qSOFA score of 0 died.

Figure 3 Severity of illness scores and mortality. For patients with COVID-19 presenting to the emergency department (ED), (A) displays the receiver operating characteristic curves for the outcome of death for the acute physiology and chronic health evaluation (APACHE) II score (AUC 0.851), simplified acute physiology score (SAPS) II (AUC 0.911), sequential organ failure assessment (SOFA) score (AUC 0.823), and quick SOFA (qSOFA) score (AUC 0.801); (B) displays the percent of patients who died by APACHE-II score at presentation to the ED; (C) displays the percent of patients who died by qSOFA score at presentation to the ED.

Discussion

This single-center cohort study found that, among all patients with COVID-19 admitted from the ED, the APACHE-II score more accurately predicted subsequent ICU admission than the SOFA, SAPS-II, or qSOFA score. Notably, both the APACHE-II and SOFA performed well using previously described modifications that obviated the need for frequently unavailable arterial blood gas measurements. We substituted S/F for the SOFA P/F value (11), eliminated the PaO₂ term from the APACHE-II (12), and used a serum bicarbonate substitution for the APACHE-II pH term (13). In this cohort, patients with either a low APACHE-II score (<8) or qSOFA score (0) did not require ICU admission. All scores accurately predicted mortality. As previously reported, advanced age and history of diabetes mellitus were associated with poor outcomes. While this cohort did not identify non-Caucasian race as an independent risk factor for ICU admission or death among this cohort of patients hospitalized with COVID-19, the proportion of non-Caucasian requiring hospitalization was twice as high as observed in historical, institutional cohorts (16,17). The finding that APACHE-II scores can accurately predict subsequent ICU admission has two basic applications.

First, when making decisions regarding allocation of hospital resources, the APACHE-II score or qSOFA may identify patients with COVID-19 at low risk for ICU admission and death, who might be safely treated in lower acuity environments. Conversely, patients with COVID-19 and high APACHE-II scores presenting to facilities with limited ICU services might benefit from early transfer to a tertiary facility. APACHE-II score may have performed better than qSOFA, SOFA, and SAPS-II because APACHE-II incorporates a broader assessment of chronic comorbidities, which may influence both the severity of COVID-19 and an individual’s physiologic reserve. It is notable that, while hypoxemia at the time of presentation is highly associated with the need for ICU admission, six patients on room air during the first 24 hours of admission required ICU care during their hospitalizations. The APACHE-II score predicted an increased risk of ICU admission in all six cases.

Second, a significant challenge for designing randomized therapeutic trials in COVID-19 is that the majority of patients will improve without treatment, which threatens to dilute outcome events, increases the required sample size, and increases the number of patients exposed to toxicities without potential benefit. Clinical trial outcomes such as death and mechanical ventilation occur predominantly in ICU patients. Thus, enrolling COVID-19 patients at risk for ICU admission into early treatment trials may prognostically enrich trials by increasing event rates, allowing smaller sample sizes, and improving the benefit/risk ratio for participants.

This study has several considerations. All studies are prone to bias, but this was mitigated through the use of objective inclusion criteria (selection bias), data collection while blinded to patient outcome (outcome bias), pre-specified modeling approaches (over-fitting bias), and sample size determination to avoid type II error. Conduct at a single tertiary care center, small sample size, and lack of a validation cohort may, however, limit generalizability. Additionally, nearly 50% of patients did not have a C-reactive protein, ferritin, or d-dimer measured at admission, which precluded additional biomarker analysis. If validated in larger cohorts, the ability of APACHE-II scores to predict ICU admission and mortality could have significant implications for patient care and clinical trials during the ongoing COVID-19 pandemic.

Conclusions

The APACHE-II score calculated within 24 hours of admission accurately predicts the eventual need for ICU care, and no patients with APACHE-II score <8 or qSOFA score of zero required ICU care. This finding could allow for safe triaging of patients to alternative care sites at times of high healthcare resource utilization and prognostically enrich future therapeutic clinical trials.

Acknowledgments

Funding: This work was supported by CTSA award No. UL1TR000445 (EMW, BIR) from the National Center for Advancing Translational Sciences. EMW was supported by NIH T32HL087738. CML was supported in part by NCI P30-CA086485. JDC was supported in part by the NIH (2T32HL087738-12, K12HL133117). MWS was supported in part by the National Heart Lung and Blood Institute (K23HL143053). Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at http://dx.doi.org/10.21037/jeccm-20-92

Data Sharing Statement: Available at http://dx.doi.org/10.21037/jeccm-20-92

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jeccm-20-92). EMW, JDC, and MWS report grants from the National Heart Lung and Blood Institute during the conduct of the study. EMW and BIR report grants from the National Center for Advancing Translational Sciences during the conduct of the study. CML reports grants from the National Cancer Institute during the conduct of the study. CML also reports grants from Novartis, grants from Xcovery, personal fees from Foundation Medicine, personal fees from Pfizer, personal fees from Novartis, personal fees from Astra-Zeneca, personal fees from Genoptix, personal fees from Sequenom, personal fees from Ariad, personal fees from Takeda, personal fees from Blueprints Medicine, personal fees from Cepheid, personal fees from Achilles, personal fees from Genentech, and personal fees from Eli Lilly outside the submitted work. BIR also reports grants from Pfizer, grants from Merck, grants from Corvus, grants from GNE/Roche, grants from Peloton, grants from Aveo, grants from Astra-Zeneca, grants from Bristol-Meyers Squib, personal fees from Aveo, personal fees from Novartis, personal fees from Synthorx, personal fees from Peloton, personal fees from Compugen, personal fees from Surface oncology, personal fees from 3DMedicines, personal fees from Arravive, and personal fees from Alkerines outside the submitted work. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was reviewed and approved by the Vanderbilt University Medical Center Institutional Review Board (IRB#200537) with waiver of informed consent.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996;22:707-10. [Crossref] [PubMed]
2. Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993;270:2957-63. [Crossref] [PubMed]
3. Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818-29. [Crossref] [PubMed]
4. Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med 2009;37:1649-54. [Crossref] [PubMed]
5. Papachristou GI, Muddana V, Yadav D, et al. Comparison of BISAP, Ranson's, APACHE-II, and CTSI scores in predicting organ failure, complications, and mortality in acute pancreatitis. Am J Gastroenterol 2010;105:435-41. [Crossref] [PubMed]
6. Goel A, Pinckney RG, Littenberg B. APACHE II predicts long-term survival in COPD patients admitted to a general medical ward. J Gen Intern Med 2003;18:824-30. [Crossref] [PubMed]
7. Simpson CE, Sahetya SK, Bradsher RW 3rd, et al. Outcomes of Emergency Medical Patients Admitted to an Intermediate Care Unit With Detailed Admission Guidelines. Am J Crit Care 2017;26:e1-10. [Crossref] [PubMed]
8. Zou X, Li S, Fang M, et al. Acute Physiology and Chronic Health Evaluation II Score as a Predictor of Hospital Mortality in Patients of Coronavirus Disease 2019. Crit Care Med 2020;48:e657-e665. [Crossref] [PubMed]
9. Lagunas-Rangel FA. Neutrophil-to-lymphocyte ratio and lymphocyte-to-C-reactive protein ratio in patients with severe coronavirus disease 2019 (COVID-19): A meta-analysis. J Med Virol 2020. [Crossref] [PubMed]
10. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:762-74. [Crossref] [PubMed]
11. Pandharipande PP, Shintani AK, Hagerman HE, et al. Derivation and validation of Spo2/Fio2 ratio to impute for Pao2/Fio2 ratio in the respiratory component of the Sequential Organ Failure Assessment score. Crit Care Med 2009;37:1317-21. [Crossref] [PubMed]
12. Chen W, Janz DR, Shaver CM, et al. Clinical Characteristics and Outcomes Are Similar in ARDS Diagnosed by Oxygen Saturation/Fio2 Ratio Compared With Pao2/Fio2 Ratio. Chest 2015;148:1477-83. [Crossref] [PubMed]
13. Okazaki H, Shirakabe A, Hata N, et al. New scoring system (APACHE-HF) for predicting adverse outcomes in patients with acute heart failure: evaluation of the APACHE II and Modified APACHE II scoring systems. J Cardiol 2014;64:441-9. [Crossref] [PubMed]
14. Team RC. R Core Team. R: A language and environment for statistical computing. Foundation for Statistical Computing 2013.
15. Robin X, Turck N, Hainard A, et al. pROC: display and analyze ROC curves. R package version 2018;1.
16. Self WH, Semler MW, Wanderer JP, et al. Balanced Crystalloids versus Saline in Noncritically Ill Adults. N Engl J Med 2018;378:819-28. [Crossref] [PubMed]
17. Semler MW, Self WH, Wanderer JP, et al. Balanced Crystalloids versus Saline in Critically Ill Adults. N Engl J Med 2018;378:829-39. [Crossref] [PubMed]