Validation of a new risk score system for non-variceal upper gastrointestinal bleeding

Abstract

Background

Recently, a new international bleeding score was developed to predict 30-day hospital mortality in patients with upper gastrointestinal bleeding (UGIB). However, the efficacy of this newly developed scoring system has not been extensively investigated. We aimed to validate a new scoring system for predicting 30-day mortality in patients with non-variceal UGIB and determine whether a higher score is associated with re-bleeding, length of hospital stay, and endoscopic failure.

Methods

A retrospective study was performed on 905 patients with acute non-variceal UGIB who were examined in our hospital between January 2013 and December 2017. Baseline characteristics, endoscopic findings, re-bleeding, admission, and mortality were reviewed. The 30-day mortality rate of the new international bleeding risk score was calculated using the receiver operating characteristic curves and compared to the pre-endoscopy Rockall score, AIMS65, Glasgow Blatchford score, and Progetto Nazionale Emorragia Digestiva score. To verify the variable for the 30-day mortality of the new scoring system, we performed multivariate logistic regression using our data and further analyzed the score items.

Results

The new international bleeding scoring system showed higher receiver operating characteristic (ROC) curve values in predicting mortality (area under ROC curve 0.958; [95% confidence interval (CI)]), compared with such as AIMS65 (AUROC, 0.832; 95%CI, 0.806–0.856; P < 0.001), PNED (AUROC, 0.865; 95%CI, 0.841–0.886; P < 0.001), Pre-RS (AUROC, 0.802; 95%CI, 0.774–0.827; P < 0.001), and GBS (AUROC, 0.765; 95%CI, 0.736–0.793; P < 0.001). Multivariate analysis was performed using our data and showed that the 30-day mortality rate was related to multiple comorbidities, blood urea nitrogen, creatinine, albumin, syncope at first visit, and endoscopic failure within 24 h during the first admission. In addition, in the high-score group, relatively long hospital stay, re-bleeding, and endoscopic failure were observed.

Conclusion

This is a preliminary report of a new bleeding score which may predict 30-day mortality better than the other scoring systems. High-risk patients could be screened using this new scoring system to predict 30-day mortality. The use of this scoring system seemed to improve the outcomes of non-variceal UGIB patients in this study, through proper management and intervention.

Background

Acute upper gastrointestinal bleeding (AUGIB) is a common medical emergency, with an incidence of 84–160 cases per 100,000 individuals and a mortality rate of approximately 10% [1]. More than 70% of AUGIB cases are non-variceal UGIB (NVUGIB), and peptic ulcer bleeding is the main cause of NVUGIB. Despite the development of endoscopic therapies and pharmacological management, NVUGIB is still associated with considerable rates of mortality and morbidity, and high medical expenses [2,3,4,5].

The International Consensus Recommendations on the management of patients with NVUGIB recommend “early risk stratification” using validated prognostic scales [6]. A number of scoring systems have been devised to predict the outcome of patients with AUGIB, including the Rockall (RS) [7], pre-endoscopic Rockall (pre-RS) [8], Glasgow-Blatchford (GBS) [ 9], AIMS65 [10, 11], and ProgettoNazionaleEmorragiaDigestiva (PNED) [12] scores. There have been many studies on the outcomes of UGIB and continued study on which scoring systems can better predict mortality. However, limited research has been performed on a unified opinion or a new superior scoring system. Recently, based on an international multicenter study, a newly developed international bleeding score (INBS) was used to predict the 30-day hospital mortality in patients with AUGIB. In the previous study, INBS well predicted the 30-day mortality in UGIB patients compared to the other scoring systems [13]. In this study, we aimed to evaluate whether INBS is effective in predicting 30-day mortality in NVUGIB patients and to determine whether re-bleeding and endoscopic treatment failure were high in the patients with a relatively high INBS score.

Methods

Patients and data collection

We retrospectively included patients who visited the emergency room (ER) for AUGIB at the Chungnam National University Hospital in Daejeon, Korea from January 2013 to December 2017. Patients older than 18 years old who presented with NVUGIB symptoms (melena, hematemesis, coffee-ground vomiting, and/or hematochezia) were included in the study. The exclusion criteria for this study were as follows: patients with lower GI, obscure GI, and variceal bleeding; those with post procedural complications; and those who were lost to follow-up after 30 days from visiting the ER.

Data were collected for each patient by reviewing their medical chart. The data included demographic data, comorbidities, chief complaints at first visit, Glasgow Coma score at the time of the emergency room visit, diagnosis including cause of bleeding, presence of endoscopic treatment, outcomes of endoscopic treatment, 30-day in-hospital mortality, length of hospital stay, hemodynamics, and laboratory result. On the basis of these data, the pre-endoscopic Rockall, Glasgow Blatchford, AIMS65, and INBS were calculated on the time of admission. The INBS is measured based on age, comorbidity, and blood test and is weighted according to severity of each category (Table 1). Three endoscopists (HY Jeong, JK Sung, and HS Moon) with more than 15 years of experience in reviewing endoscopic findings using the picture archiving communication system (PACS) classified the findings as gastric ulcers (GUs), duodenal ulcers (DUs), Dieulafoy’s lesions, Mallory-Weiss syndrome, esophageal ulcers, angio-dysplasia, hemorrhagic gastritis, acute gastric mucosal lesion, and cancer-related bleeding. The data for each scoring system were entered by one researcher (MS Kim). Some verified scoring systems, for example, pre-endoscopy Rockall score, AIMS65, Glasgow Blatchford score, are automatically calculated in the electronical program in our hospital where this study was conducted. However, the new risk score system is not currently fully validated; thus, it was calculated manually. Moreover, to minimize errors and bias, the first author of this paper (MS Kim) managed it.

Table 1 International bleeding risk score for predicting 30-day mortality

Definitions

The grading of overall health comorbidity was performed according to the American Society of Anesthesiologists (ASA) classification [14]. As in previous studies [13], we categorized the patients into the high-risk group or the low-to-moderate-risk group using a cut-off INBS score; the high-risk group included patients with an INBS score > 7, whereas the low-to-moderate-risk group included patients with an INBS score ≤ 7.

The primary outcome was in-hospital mortality at 30 days, defined as any death occurring during hospitalization. Secondary outcomes were assessed for recurrence of bleeding (re-bleeding), duration of hospital stay, and endoscopic hemostasis failure at the first visit in the high-risk group.

Re-bleeding was characterized as fresh hematemesis and/or melena associated with the development of shock (pulse > 100 beats/min, systolic blood pressure < 100 mmHg), or a reduction in the hemoglobin concentration > 2 g/dL for > 24 h. This also included cases of re-bleeding upon repeat endoscopy [15].

The hospitalization period was defined as the date from visiting the ER to the date of discharge. Endoscopic hemostasis failure at first visit was defined as instability at the time of first visit and inability to perform endoscopic hemostasis or vascular embolization after angiography or surgery due to endoscopic hemostasis failure.

Statistical analyses

We compared each scores’ discriminative ability to predict 30-day mortality by calculating the area under the receiver operating characteristic curves (AUROCs) with 95% confidence intervals (CIs). The optimal cut-off score for predicting very high-risk patients in each scoring system was determined by the maximum Youden index, with a sensitivity and specificity of 95%. In addition, the performance of the scoring systems was assessed by calculating the sensitivity, specificity, percentage of patients classified as high risk, and the mortality rate among them.

Based on the risk factors previously mentioned in the scoring systems, univariate and multivariate logistic regression analyses were performed to evaluate the risk factors for predicting the 30-day mortality of NVUGIB patients visiting our hospital. A P < 0.05 was regarded as statistically significant. This was to ensure that the risk factors mentioned in the other scoring systems were actually a significant risk factor in our database. All statistical analyses were performed with the SPSS (version21.0, SPSS Inc., Chicago, IL, USA) and MedCalc (version18.11,MedCalc®, MedCalc Software, Belgium).

Results

Data of 1118 patients aged ≥18 years who visited the ER owing to UGIB for > 5 years were reviewed. We excluded 213 patients from the study for the following reasons: 23 patients had non-UGIB, 155 had variceal bleeding, 15 had postprocedure-associated bleeding, and 20 had obscure GI bleeding. In total, 905 patients were selected for evaluation. Of these, 131 and 774 patients were included in the high- and low-to-moderate-risk groups, respectively (Fig. 1).

Fig. 1

Flow chart of patients enrolled in this study. GI, gastrointestinal

Patients’ characteristics

Of the 905 included patients, 674 (74.5%) were men and 231 (25.5%) were women, with a mean age of 65.4 ± 14.7 years. Upper GIT endoscopy was performed in all included patients. Seven hundred ten patients (78.4%) underwent endoscopy within 12 h during the admission in the emergency room, 160 patients (17.7%) underwent endoscopy within 12–24 h and 35 patients (3.9%) underwent endoscopy after 48 h. There were 35 patients who did not undergo endoscopy within 48 h of the first visit; most endoscopy procedures were performed after intensive care unit (ICU) hospitalization due to unstable vital signs. Among the total patients who underwent endoscopy, 735 received endoscopic hemostasis therapy; of these, patients underwent surgery (n = 11), trans-arterial embolization due to failure of the endoscopic treatment (n = 32), or both (n = 2). Among the patients who received endoscopic hemostasis, 128 had re-bleeding during the second endoscopy. Emergency endoscopic procedures were performed in most cases where patients had rebleeding during hospitalization and rebleeding after discharge. Thus, urgent endoscopic procedures within 12 h were performed in patients with rebleeding, and the mean average endoscopy timing in relation to rebleeding is 2.75 h.

Calculation of pre-RS, GBS, AIMS65, PNED, and INBS was possible in all cases. Table 2 shows the patients’ characteristics, chief complaint at first visit, treatment, and outcomes. The comorbidity scores were classified as ASA scores. The commonest endoscopic diagnosis was GU (67.8%), and the second commonest was DU (19.3%). There were 467 (51.6%) patients admitted to the hospital within 8 h. The average length of hospital stay for all patients was approximately 8.7 days. Among the 905 enrolled patients, only 31 patients (3.4%) were hospitalized for > 30 days A total of 44 patients (4.9%) died within 30 days in hospital. A total of 44 patients died in the hospital within 30 days. Of these, 14 had a direct association with non-variceal upper gastrointestinal bleeding, 12 died of heart failure, 11 of septic shock, 4 of liver failure, 2 of cerebral infarction, and 1 of cerebral hemorrhage. The mean INBS scores in the mortality and survivor groups were 8.38 ± 3.12 and 3.86 ± 2.56 (P value = 0.027), respectively.

Table 2 Baseline characteristics, treatment, and clinical outcomes of the study population

Comparison of bleeding scores’ discriminative ability to predict the 30-day mortality

In this study, INBS had the highest discriminative ability (area under the receiver operating characteristic (AUROC) curve 0.958 [95% confidence interval (CI), 0.943–0.970]) in predicting mortality within 30 days compared with the AIMS65 (0.832; P < 0.001), PNED score (0.865; P < 0.001), Pre-RS (0.802; P < 0.001), and GBS (0.765; P < 0.001). The cut-off score was > 7 with a sensitivity of 97.73% and a specificity of 89.79% (Table 3 and Fig. 2). Table 4 shows the discriminative ability of the evaluated scoring systems to predict length of hospital stay, rebleeding, and endoscopic hemostasis failure.

Table 3 Predictive and discriminative abilities for identification of patients at high-risk of mortality in 30 days

Fig. 2

Comparison of scoring systems in the prediction of 30-day mortality (n = 905). AUROC, area under receiver operating characteristic curve [95% CI]; INBS, international new bleeding score; Pre-RS, pre-endoscopic Rockall score; GBS, Glasgow Blatchford score; PNED, Progetto Nazionale Emorragia Digestiva score

Table 4 Discriminative ability of the evaluated scoring systems

The cut-off value was used to organize the patients into the high- and low-to-moderate-risk groups. A total of 131 patients (14.5%) were in the high-risk group; of these, 43 (32%) died within 30 days. The low-to-moderate-risk group comprised 774 patients (85.5%).

Multivariate analysis for 30-day mortality

On the basis of the risk factors mentioned in the scoring system for UGIB patients, we performed logistic regression to identify predictors associated with mortality in patients who visited our hospital. In the univariate regression analysis, the variables that were meaningful were male sex, old age, smoking, ASA score of 4, hypertension, acute respiratory distress syndrome (ARDS), disseminated malignancy, liver cirrhosis, sepsis, disseminated intravascular coagulation, systolic blood pressure, heart rate, hemoglobin, platelet count, blood urea nitrogen (BUN), creatinine, international normalized ratio, syncope at first visit, endoscopic failure at first admission in 48 h, endoscopic hemostasis failure, and re-bleeding at the second endoscopy. Multivariate regression analysis was performed with the abovementioned variables that were significant in the univariate regression analysis (Table 5). Hypertension and systolic blood pressure could be duplicated, and only one was added. The multivariate analysis revealed that an ASA score of 4, ARDS, disseminated malignancy, creatinine, albumin, syncope at first visit, and endoscopic failure within 24 h during the first admission were associated with 30-day mortality.

Table 5 Univariate and multivariate analyses for 30-day mortality (n = 905)

Re-bleeding and length of hospital stay in the high-risk group

An INBS cut-off value > 7 was used to categorize patients into the high-score group (131 patients, 14.4%) and low-score group (774 patients, 85.5%). The high-score group had a relatively longer length of hospital stay and higher re-bleeding and endoscopic hemostasis failure rates than the low-score group (Table 6).

Table 6 Long-term hospital stay and occurrence of re-bleeding between high-risk group and low-to-moderate risk group in INBS

Discussion

In this study, we evaluated whether INBS is effective in predicting 30-day mortality in NVUGIB patients and its utility in predicting rebleeding or hospitalization duration. INBS was superior to other pre-endoscopy risk scoring systems in predicting 30-day mortality.

AUGIB is a common medical emergency associated with high morbidity and 30-day mortality rates [1, 16]. More than 70% of AUGIB cases are NVUGIB, with GUs or DUs being the commonest. Mallory-Weiss syndrome, Dieulafoy’s ulcer, angiodysplasia, and cancer-related bleeding are also causes of NVUGIB [17]. While therapies such as Helicobacter pylori eradication therapy and proton pump inhibitors might be expected to reduce peptic ulcers and decreased the mortality rate of NVUGIB patients, studies show that the mortality rate is still high at 6 to 14% due to population aging and the use of anti-platelet drugs [18,19,20,21]. In these NVUGIB patients, timely hemostatic endoscopic procedures are important for survival improvement. For a successful treatment, assessing the hemodynamic status and appropriate risk measurement are necessary. Thus, classifying high-risk NVUGIB patients who have arrived at the ER using a highly efficient scoring system is important to help predict prognosis and direct appropriate treatment [7].

A variety of indicators (AIMS65 [10, 11], GBS [9], Pre-RS [8], PNED [12]) have been developed and evaluated to assess the risk of NVUGIB. Hyett et al. found that the AIMS65 score was superior to GBS in predicting death, but the GBS was better in predicting the need and the number of packed red blood cell transfusions [22]. In a cohort study of 424 participants, AIMS65 was also superior to GBS and Pre-RS in predicting hospital mortality, ICU admission, and loss of conscious [23]. In a European study, the usefulness of GBS, Pre-RS, and AIMS65 in patients with UGIB was assessed; however, there was no difference in mortality or re-bleeding frequency among the three scoring systems, and GBS was the best predictor of transfusion [24]. In another European study of 309 patients with UGIB, AIMS65, GBS, and Pre-RS were reported to be similar when predicting patient mortality; however, the need for endoscopic intervention was better predicted by AIMS65 and GBS [25, 26]. In a study on the utility of GBS and AIMS65 conducted in the United States and involving 165 patients with NVUGIB aged ≥65 years, GBS was superior to AIMS65 in predicting mortality [15]. The evaluation of the usefulness and predictability of the scoring system for assessing the prognosis of NVUGIB varies according to each study. The INBS¹³ scoring system that we have developed provides clear criteria for the 30-day mortality risk of patients with NVUGIB and has been shown to predict mortality better than the previous scoring systems. It is a predictive scoring system similar to the Pre-RS and PNED scores. However, INBS provides clear criteria for cirrhosis, differentiation of ASA scores according to hemodialysis, and blood tests to assess liver and renal functions. In contrast to previous scoring systems, the INBS included medical history including liver, renal and heart failure, in addition to the objective metabolic and biochemical indicators which we have described.

Our research suggests several strengths based on this new scoring system. First, this study validated INBS against the Pre-RS, GBS, AIMS65, and PNED risk stratification scores for predicting 30-day mortality. INBS was superior to other pre-endoscopy risk scores in predicting 30-day mortality. AIMS65, PNED, and Pre-RS showed higher mortality predictions with an AUROC > 0.8, but it had lower values compared to INBS. Secondly, variables considered as risk factors in each scoring system were confirmed in this study. In the multivariate analysis, an ASA score of 4, ARDS, disseminated malignancy, creatinine, albumin, syncope at first visit, and failure of endoscopic treatment within 24 h of the first visit were identified as risk factors for death in NVUGIB patients. The risk of death due to underlying disease factors, aggravation of liver and kidney diseases, and failure of endoscopic treatment at the first visit were confirmed. The significant death risk factors in this study were similar to the death risk factors already reflected in INBS. INBS has proposed criteria to identify liver and renal failures through liver cirrhosis, decreased albumin levels, and increased BUN and creatinine levels. The severity of the underlying disease was scored using the ASA classification. Thirdly, we classified the patients into the high- and low-to-moderate-risk groups based on the INBS cutoff value of > 7. The cut-off value was the same to that published in a previous study of ≥8 for the high-risk group [16]. In our study, we showed that the high INBS group had a relatively long hospital stay and high re-bleeding and endoscopic hemostasis failure rates. The cause of death from bleeding was one of the predictable outcomes if the patient was unable to tolerate hemostatic treatment due to a poor general condition or bleeding from cancer and massive bleeding from the large vessels. Therefore classification of patients using INBS could be used as a tool to increase the probability of successful hemostasis and to shorten the hospitalization period through rapid endoscopic treatment and intensive monitoring.

In Korea, tertiary medical care institutions properly and actively perform endoscopy in patients. Therefore it would be helpful for the doctors who perform such procedures to have a classification standard score to help determine the appropriate timing of endoscopy and to predict the prognosis of NVUGIB patients. Using INBS, the identification of patients with a high risk of death is possible, allowing targeted management and interventions that may improve outcome.

This study has some limitations. First, the present investigation was a retrospective, single-center study, which can be a confounding factor. Therefore, a larger sample size and a prospectively designed study are needed to confirm the effectiveness of INBS. Second, this is an observational study, which focused on a high-risk mortality factor and classified high-risk patients. However, the criteria for low-risk patients with low scores were not established, and neither the criteria for the need for endoscopic treatment nor those for outpatient follow-up were presented. Given the nature of the tertiary medical institutions, most patients visiting the ER with AUGIB are often referred from other medical institutions because of the severity of the disease. Thus, doctors often think that endoscopy should be performed or patients want to undergo endoscopy. Therefore, examining our study subjects is important to confirm the suitability of the score in selecting low-risk patients to receive outpatient treatment without endoscopy. In addition, the INBS scoring system is limited in its use as a criterion for selecting low-risk groups, because it is weighted towards identifying patients who are at risk of sudden aggravation or death from liver or kidney disease. Although this study has several limitations, we think that it has laid the foundation for larger, multi-center, prospective studies of the utility of the INBS scoring system. Specifically, such studies would be designed to investigate whether carrying out early endoscopy, and second-look endoscopy on the basis of the degree of risk identified by this new scoring system can improve patient outcomes.

Conclusion

This is a preliminary report of a new bleeding score which may predict 30-day mortality better than the other scoring systems. High-risk patients could be screened using this new scoring system to predict 30-day mortality, longer hospital admission, re-bleeding, and endoscopic hemostasis failure. The use of this scoring system seemed to improve the outcomes of non-variceal UGIB patients in this study, through proper management and intervention.