Automated Machine Learning in Predicting 30-Day Mortality in Patients with Non-Cholestatic Cirrhosis
by
, , ,
Chenyan Yu
1,2,3,†, 
Yao Li
1,2,†,
Minyue Yin
1,2,
Jingwen Gao
1,2,
Liting Xi
1,2,
Jiaxi Lin
1,2,
Lu Liu
1,2,
Huixian Zhang
3,
Airong Wu
1,2,
Chunfang Xu
1,2,
Xiaolin Liu
1,2,
Yue Wang
4,* and
Jinzhou Zhu
1,2,* 1
Department of Gastroenterology, The First Affiliated Hospital of Soochow University, 188 Shizi Street, Suzhou 215006, China
2
Suzhou Clinical Center of Digestive Diseases, Suzhou 215000, China
3
Department of Gastroenterology, Dushu Lake Hospital Affiliated to Soochow University, Suzhou 215000, China
4
Department of Hepatology, The Fifth People’s Hospital of Suzhou, Suzhou 215000, China
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
J. Pers. Med. 2022, 12(11), 1930; https://doi.org/10.3390/jpm12111930
Submission received: 15 October 2022
/
Revised: 9 November 2022
/
Accepted: 18 November 2022
/
Published: 19 November 2022
(This article belongs to the Special Issue Towards Personalized Medicine for Chronic Liver Disease)
Abstract
:Objective: To evaluate the feasibility of automated machine learning (AutoML) in predicting 30-day mortality in non-cholestatic cirrhosis. Methods: A total of 932 cirrhotic patients were included from the First Affiliated Hospital of Soochow University between 2014 and 2020. Participants were divided into training and validation datasets at a ratio of 8.5:1.5. Models were developed on the H2O AutoML platform in the training dataset, and then were evaluated in the validation dataset by area under receiver operating characteristic curves (AUC). The best AutoML model was interpreted by SHapley Additive exPlanation (SHAP) Plot, Partial Dependence Plots (PDP), and Local Interpretable Model Agnostic Explanation (LIME). Results: The model, based on the extreme gradient boosting (XGBoost) algorithm, performed better (AUC 0.888) than the other AutoML models (logistic regression 0.673, gradient boost machine 0.886, random forest 0.866, deep learning 0.830, stacking 0.850), as well as the existing scorings (the model of end-stage liver disease [MELD] score 0.778, MELD-Na score 0.782, and albumin-bilirubin [ALBI] score 0.662). The most key variable in the XGBoost model was high-density lipoprotein cholesterol, followed by creatinine, white blood cell count, international normalized ratio, etc. Conclusion: The AutoML model based on the XGBoost algorithm presented better performance than the existing scoring systems for predicting 30-day mortality in patients with non-cholestatic cirrhosis. It shows the promise of AutoML in its future medical application.
1. Introduction
Cirrhosis is a leading cause of morbidity and mortality across the world, which is characterized by a systemic pro-inflammatory milieu consisting of persistent liver inflammation, extracellular matrix remodeling, and the accumulation of collagen in liver tissue [1]. Etiologically, viral hepatitis, especially hepatitis B viral (HBV), has been the leading pathogeny of cirrhosis. In addition, several kinds of cirrhosis were associated with cholestasis, such as primary sclerosing cholangitis (PSC) and primary biliary cholangitis (PBC) [2]. Complications mainly include ascites, portal hypertensive gastrointestinal bleeding, jaundice, coagulopathy, and hepatic encephalopathy which may recur with increasing frequency after the initial presentation, and most patients die within a median time of approximately 2 years [3,4]. Most studies support that early intervention in stabilized cirrhosis may delay its progression to the decompensated stage [5].
Several prediction models for the prognosis of liver cirrhosis have been proposed. The Child-Turcotte-Pugh score (CTP), including serum levels of bilirubin and albumin, prothrombin time, degree of ascites, and severity of hepatic encephalopathy, was initially introduced to predict mortality in patients with cirrhosis who underwent surgery [6]. It consists of two subjective indicators, which were prone to bias during application. The model for end-stage liver disease (MELD) score was validated as a predictor of allocation of organs for liver transplantation, rather than originally used to assess mortality in patients undergoing trans-jugular intrahepatic portosystemic shunt (TIPS) [7,8]. Even though these above models were classical, they ignored the effect of cholestasis on lipid metabolism in patients with cirrhosis.
Machine learning (ML) is a scientific discipline that emphasizes efficient computing algorithms, including supervised, unsupervised, semi-supervised, and reinforcement learning, which are widely used in medicine [9]. Traditional ML includes support vector machine (SVM), gradient boosting machine (GBM), extreme gradient boosting (XGBoost), etc. Automated machine learning (AutoML) intelligently computes hundreds or even thousands of mathematical models and eventually filters out the optimal model, which is more suitable for clinicians without a computer foundation.
A series of previous studies has proven that clinical models based on machine learning performed better than models based on traditional logistic regression. There were no previous reports concerning AutoML and cirrhosis, thus we conducted this hospital-based case-control study to develop AutoML models for predicting 30-day mortality in patients with non-cholestatic cirrhosis. On one hand, we evaluate the feasibility of AutoML in the management of chronic liver disease. On the other hand, we observe the performance of AutoML in clinical modeling based on electronic health records. In this study, multiple models, based on various AutoML algorithms, were developed and compared with the existing scoring systems.
2. Materials and Methods
2.1. Study Design
This was a hospital-based cohort study involving 932 patients with non-cholestatic cirrhosis from the First Affiliated Hospital of Soochow University between 2014 and 2020. A random grouping method was used to divide all participants into the training and validation datasets according to the ratio of 8.5:1.5, consisting of 792 and 140 patients, respectively. The outcome was 30-day mortality since hospitalization. All participants had signed informed consent for the medical data used in the study. According to the principles of the Declaration of Helsinki, the study was approved by local Institutional Review Boards.
A series of AutoML algorithms on the H2O platform were used for modelling in the training dataset. H2O, an open source and scalable platform for machine learning, provided multiple algorithms and used K-fold cross-validation to develop and validate models [10]. Models were evaluated by area under receiver operating characteristic (ROC) curves (AUC) and compared with the existing scorings, including the model of end-stage liver disease (MELD) score, the MELD-Na score, and the albumin-bilirubin (ALBI) score, in the training and validation datasets. The flowchart is plotted in Figure 1.
2.2. Inclusion and Exclusion Criteria
Subjects included had clear evidence of non-cholestatic liver cirrhosis either by liver histology or by clinical presentations, liver function tests, and medical imaging techniques. Exclusion criteria included the presence of solid organ transplantation, malignant neoplasm, PBC, and lack of complete medical records. The diagnosis of PBC is based on the following criteria [2]: (1) biochemical evidence of cholestasis with an elevation of alkaline phosphatase activity; (2) presence of antimitochondrial antibody (AMA); (3) histopathologic evidence of nonsuppurative cholangitis and destruction of small or medium-sized bile ducts if a biopsy is performed.
2.3. Clinical Data Collection
The data of all participants were retrospectively enrolled from the Department of Gastroenterology and Hepatology, The First Affiliated Hospital of Soochow University. Demographic information included gender, age, body mass index (BMI), history of hypertension, history of diabetes, etiology of cirrhosis, and complications of cirrhosis. Laboratory parameters include white blood cell count (WBC), platelet count (PLT), total bilirubin (TBIL), creatinine, alanine aminotransferase (ALT), aspartate aminotransferase (AST), albumin, sodium, total cholesterol (TC), triglycerides (TG), high-density lipoprotein cholesterol (HDL-C), low-density lipoprotein cholesterol (LDL-C), prealbumin, prothrombin time (PT), and international standard ratio (INR). Biochemical analyses were all available from a Hitachi 7600 Auto-Analyzer (Hitachi, Tokyo, Japan) or an Abbott-Architect Immunoanalyzer (Abbott Laboratories, Abbott Park, IL, USA). According to survival conditions 30 days after admission, participants were separately divided into two groups, namely the death and survival groups. The calculation methods of MELD score [7], MELD-Na score [11], and ALBI score [12] were as follows:
MELD = 3.78 × ln[serum bilirubin (mg/dL)] + 11.2 × ln[INR] + 9.57 × ln[serum creatinine mg/dL] + 6.43.
MELD-Na = MELD-Na − [0.025 × MELD × (140 − Na)] + 140, where the serum sodium concentration is bound between 125 and 140 mmol/L.
ALBI = 0.66 × log[serum bilirubin (mg/dL)] − 0.085 × protein (mg/dL).
2.4. Statistics Analysis
Data analysis was performed by R (version 4.1.0, R Foundation for Statistical Computing, Vienna, Austria), and SPSS (version 24.0, SPSS Inc., Chicago, IL, USA). Continuous variables were expressed as median (Q1–Q3) or mean ± standard, and categorical variables were expressed by frequencies and percentages. Binary logistic regression analysis and automated machine learning based on the H2O package were used to establish the predictive model for non-cholestatic cirrhosis. The discrimination of the model was assessed by using receiver operating curves (ROC). Our study utilized three methods for model interpretation: the SHapley Additive exPlanations (SHAP) analysis, Partial Dependency Plots (PDP), and Local Interpretable Model Agnostic Explanation (LIME), which represented how the features affect the output of the model’s prediction. A two-sided p < 0.05 was considered statistically significant.
3. Results
3.1. Characteristics of the Patients
A total of 932 cirrhotic patients were enrolled in this study, including 136 in the non-survival group and 796 in the survival group. The 30-day mortality rate was 14.6%. Further details on the characteristics of patients for 30-day mortality are given in Table 1.
3.2. Models Based on AutoML Algorithms
In the validation dataset, the AutoML model based on the XGBoost algorithm showed the best performance in discrimination (AUC = 0.888) when compared to logistic regression (AUC = 0.673), gradient boost machine (AUC = 0.886), random forest (AUC = 0.866), deep learning (AUC = 0.830), stacking (AUC = 0.850), MELD (AUC = 0.778), MELD-Na (AUC = 0.782), and ALBI (AUC = 0.662) as shown in Table 2.
3.3. Interpretation of the AutoML Model Based on XGBoost Algorithm
Figure 2 shows the ten key variables in the AutoML model based on the XGBoost algorithm. HDL-C was the foremost feature, followed by WBC, creatinine, age, PT, TC, albumin, etc.
Moreover, we plotted SHAP, which is a unified approach for explaining the outcome of any machine learning mode, to provide consistent and accurate attribution values for each feature. Figure 3 presented that the closer the values of features in the model were to 1.5, the closer the correlations were with 30-day mortality of cirrhosis.
PDP were drawn to more clearly show the relationship between various variables and mortality of cirrhosis (Figure 4). Among the key variables, creatinine, WBC, INR, and age showed upward trends with the mortality, while HDL-C and TC showed opposite trends.
As shown in Figure 5, LIME plot shows how the important variables (in the XGBoost model) contributed to the mortality based on four selected samples. For example, case #4 had a high probability of 0.68 for death due to cirrhosis as predicted by the XGBoost model. HDL-C, creatinine, TC, WBC, and age played distinct roles in the prediction.
4. Discussion
Based on the clinical data of the 932 cirrhotic patients, we have developed and validated a series of AutoML models in predicting 30-day mortality.
We found the AutoML model based on the XGBoost algorithm showed best performance of discrimination when compared to the other models and the existing scoring systems. Furthermore, we visualized the best model for interpretation using SHAP, PDP, and LIME.
In terms of the key variables for the mortality in cirrhosis, we found HDL-C was the most important variable in the XGBoost model. The level of HDL-C was associated with the severity of end-stage liver disease [13,14]. Terib et al. demonstrated that HDL-C level was a robust predictor for survival in patients with chronic liver failure [15]. Habib et al. suggested that HDL-C was significantly decreased in non-cholestatic cirrhotic veterans [16]. The mechanisms underlying the decrease in HDL-C level due to cirrhosis are unclear, but recent studies have focused on the anti-inflammatory effects of HDL-C. HDL-C can decrease cholesterol levels in cell membranes by removing intracellular lipids, thereby reducing lipid secretion of pro-inflammatory cytokines [17]. According to the published studies, we have observed that cirrhosis impaired the ability of HDL-C to inhibit lipopolysaccharide-induced (LPS) activation of the pro-inflammatory transcription factor NF-kB and subsequent production of interleukin-6 (IL-6), interleukin-8 (IL-8, and tumor necrosis factor-α (TNF-α) in monocytes [18]. In addition, a study in patients with advanced chronic liver failure showed that the addition of recombinant HDL-C to restore HDL function reduced the LPS-induced inflammatory response [19]. The evidence was confirmed in various animal models [20].
We also found that higher WBC counts at admission were associated with mortality, which was inconsistent with a previous study in the United States [21]. It is possible that cirrhotic patients who mount a leukocytosis have a higher pro-inflammatory milieu, resulting in a decrease in survival rate [22,23]. Moreover, consistent with the results of several previous studies [24,25], elevated creatinine levels were an associated risk factor for poor prognosis in cirrhosis, which is mainly related to vasodilatory mechanisms and non-vasodilatory mechanisms identified in recent years [26]. The vasodilatory mechanism is mainly related to the dilatation of visceral arterial vessels during the decompensated phase of cirrhosis, the reduction of systemic effective circulating blood volume, the reduction of cardiac output, the activation of the renin-angiotensin-aldosterone system, and the impaired renal function due to insufficient renal perfusion [27]. Lastly, various inflammatory factors, intra-abdominal hypertension, high bilirubin and bile acids, relative adrenal insufficiency, and other non-vasodilatory mechanisms also play an important role in creatinine elevation [28,29].
In addition, this study found that PT and INR values were larger in the mortality group than in the survival group, suggesting that coagulation function was a good predictive indicator of 30-day mortality in non-cholestatic cirrhosis. P. G. Northup et al. reported that the INR and PT were inextricably linked to the prognosis and progression of liver disease [30]. Both J. Li et al. [31] and T. Wu et al. [32] developed predictive models on the prognosis of chronic liver disease and also included INR, which was mainly related to the involvement of the liver in coagulation factors. This study also revealed that age was an independent risk factor for poor prognosis of cirrhosis, consistent with recent epidemiological findings [33], which may be associated with weakened immune systems in the elderly.
A good predictive model can effectively and accurately predict the prognosis of disease, and facilitate the early monitoring of high-risk patients to reduce the mortality of the disease in question. In the study, we reported the application of AutoML in predicting mortality of cirrhosis for the first time. We found the model based on the XGBoost algorithm showed the best performance of discrimination when compared to the existing scoring systems, e.g., MELD, MELD-Na, and ABLI. To overcome the drawback of the black box in machine learning, we interpreted the model using various means of visualization.
There are several limitations in this study: first, this is a single-central study that may lead to results bias and multi-central data are needed to validate the new model; second, our results are observational, which could not prove causality and may be influenced by unmeasured confounders; finally, the outcome is only designed for 30-day mortality and may require further long-term follow-up.
In this study, a series of AutoML models were developed for predicting 30-day mortality in patients with non-cholestatic cirrhosis. The model, based on the XGBoost algorithm, presented best performance when compared to existing scoring systems. It shows the promise of AutoML in its future medical application.
Author Contributions
Conceptualization, Y.W. and J.Z.; methodology, Y.L.; software, M.Y. and J.L.; validation, C.Y., J.G. and L.X.; formal analysis, L.L.; investigation, H.Z.; resources, C.Y.; data curation, A.W.; writing—original draft preparation, C.Y. and Y.W.; writing—review and editing, C.X. and X.L.; visualization, M.Y.; supervision, Y.W. and J.Z.; project administration, J.Z.; funding acquisition, Y.W. and J.Z. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by Science and Technology Plan of Suzhou City (SKY2021038) and the Youth Program of Suzhou Health Committee (KJXW2019001 and KJXW2021051).
Institutional Review Board Statement
According to the principles of the Declaration of Helsinki, the study was approved by local Institutional Review Boards.
Informed Consent Statement
Patient consent was waived due to the fact that this study is retrospective.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Bernardi, M.; Angeli, P.; Claria, J.; Moreau, R.; Gines, P.; Jalan, R.; Caraceni, P.; Fernandez, J.; Gerbes, A.L.; O’Brien, A.J.; et al. Albumin in decompensated cirrhosis: New concepts and perspectives. Gut 2020, 69, 1127–1138. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lindor, K.D.; Gershwin, M.E.; Poupon, R.; Kaplan, M.; Bergasa, N.V.; Heathcote, E.J. American Association for Study of Liver D: Primary biliary cirrhosis. Hepatology 2009, 50, 291–308. [Google Scholar] [CrossRef] [PubMed]
- D’Amico, G.; Morabito, A.; D’Amico, M.; Pasta, L.; Malizia, G.; Rebora, P.; Valsecchi, M.G. Clinical states of cirrhosis and competing risks. J. Hepatol. 2018, 68, 563–576. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- D’Amico, G.; Garcia-Tsao, G.; Pagliaro, L. Natural history and prognostic indicators of survival in cirrhosis: A systematic review of 118 studies. J. Hepatol. 2006, 44, 217–231. [Google Scholar] [CrossRef] [PubMed]
- D’Amico, G.; Pasta, L.; Morabito, A.; D’Amico, M.; Caltagirone, M.; Malizia, G.; Tine, F.; Giannuoli, G.; Traina, M.; Vizzini, G.; et al. Competing risks and prognostic stages of cirrhosis: A 25-year inception cohort study of 494 patients. Aliment. Pharm. 2014, 39, 1180–1193. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Pugh, R.N.H.; Murray-Lyon, I.M.; Dawson, J.L.; Pietroni, M.C.; Williams, R. Transection of the oe sophagus for bleeding oesophageal varice. Br. J. Surg. 1973, 60, 646–649. [Google Scholar] [CrossRef]
- Wiesner, R.; Edwards, E.; Freeman, R.; Harper, A.; Kim, R.; Kamath, P.; Kremers, W.; Lake, J.; Howard, T.; Merion, R.M.; et al. Model for end-stage liver disease (MELD) and allocation of donor livers. Gastroenterology 2003, 124, 91–96. [Google Scholar] [CrossRef] [Green Version]
- Kamath, P.S.; Kim, W.R. Advanced Liver Disease Study G: The model for end-stage liver disease (MELD). Hepatology 2007, 45, 797–805. [Google Scholar] [CrossRef]
- Deo, R.C. Machine Learning in Medicine. Circulation 2015, 132, 1920–1930. [Google Scholar] [CrossRef] [Green Version]
- Liu, G.; Lu, D.; Lu, J. Pharm-AutoML an open-source, end-to-end automated machine learning package for clinical outcome prediction. CPT Pharmacomet. Syst. Pharm. 2021, 10, 478–488. [Google Scholar] [CrossRef]
- Biggins, S.W.; Kim, W.R.; Terrault, N.A.; Saab, S.; Balan, V.; Schiano, T.; Benson, J.; Therneau, T.; Kremers, W.; Wiesner, R.; et al. Evidence-based incorporation of serum sodium concentration into MELD. Gastroenterology 2006, 130, 1652–1660. [Google Scholar] [CrossRef] [PubMed]
- Johnson, P.J.; Berhane, S.; Kagebayashi, C.; Satomura, S.; Teng, M.; Reeves, H.L.; O’Beirne, J.; Fox, R.; Skowronska, A.; Palmer, D.; et al. Assessment of liver function in patients with hepatocellular carcinoma: A new evidence-based approach-the ALBI grade. J. Clin. Oncol. 2015, 33, 550–558. [Google Scholar] [CrossRef] [PubMed]
- Cicognani, C.; Malavolti, M.; Morselli-Labate, A.M.; Zamboni, L.; Sama, C.; Barbara, L. Serum Lipid and Lipoprotein Patterns in Patients with Liver Cirrhosis and Chronic Active Hepatitis. Arch. Intern. Med. 1997, 157, 792–796. [Google Scholar] [CrossRef] [PubMed]
- Etogo-Asse, F.E.; Vincent, R.P.; Hughes, S.A.; Auzinger, G.; Le Roux, C.W.; Wendon, J.; Bernal, W. High density lipoprotein in patients with liver failure; relation to sepsis, adrenal function and outcome of illness. Liver Int. 2012, 32, 128–136. [Google Scholar] [CrossRef]
- Trieb, M.; Rainer, F.; Stadlbauer, V.; Douschan, P.; Horvath, A.; Binder, L.; Trakaki, A.; Knuplez, E.; Scharnagl, H.; Stojakovic, T.; et al. HDL-related biomarkers are robust predictors of survival in patients with chronic liver failure. J. Hepatol. 2020, 73, 113–120. [Google Scholar] [CrossRef] [Green Version]
- Habib, A.; Mihas, A.A.; Abou-Assi, S.G.; Williams, L.M.; Gavis, E.; Pandak, W.M.; Heuman, D.M. High-Density Lipoprotein Cholesterol as an Indicator of Liver Function and Prognosis in Noncholestatic Cirrhotics. Clin. Gastroenterol. Hepatol. 2005, 3, 286–291. [Google Scholar] [CrossRef]
- Dogan, I.; Karaman, K.; Sonmez, B.; Celik, S.; Turker, O. Relationship between serum neutrophil count and infarct size in patients with acute myocardial infarction. Nucl. Med. Commun. 2009, 30, 797–801. [Google Scholar] [CrossRef]
- Trieb, M.; Horvath, A.; Birner-Gruenberger, R.; Spindelboeck, W.; Stadlbauer, V.; Taschler, U.; Curcic, S.; Stauber, R.E.; Holzer, M.; Pasterk, L.; et al. Liver disease alters high-density lipoprotein composition, metabolism and function. Biochim. Biophys Acta 2016, 1861, 630–638. [Google Scholar] [CrossRef] [Green Version]
- Galbois, A.; Thabut, D.; Tazi, K.A.; Rudler, M.; Mohammadi, M.S.; Bonnefont-Rousselot, D.; Bennani, H.; Bezeaud, A.; Tellier, Z.; Guichard, C.; et al. Ex vivo effects of high-density lipoprotein exposure on the lipopolysaccharide-induced inflammatory response in patients with severe cirrhosis. Hepatology 2009, 49, 175–184. [Google Scholar] [CrossRef]
- Thabut, D.; Tazi, K.A.; Bonnefont-Rousselot, D.; Aller, M.; Farges, O.; Guimont, M.C.; Tellier, Z.; Guichard, C.; Ogier-Denis, E.; Poynard, T.; et al. High-density lipoprotein administration attenuates liver proinflammatory response, restores liver endothelial nitric oxide synthase activity, and lowers portal pressure in cirrhotic rats. Hepatology 2007, 46, 1893–1906. [Google Scholar] [CrossRef]
- Bajaj, J.S.; Reddy, K.R.; O’Leary, J.G.; Vargas, H.E.; Lai, J.C.; Kamath, P.S.; Tandon, P.; Wong, F.; Subramanian, R.M.; Thuluvath, P.; et al. Serum Levels of Metabolites Produced by Intestinal Microbes and Lipid Moieties Independently Associated with Acute-on-Chronic Liver Failure and Death in Patients with Cirrhosis. Gastroenterology 2020, 159, 1715–1730.e1712. [Google Scholar] [CrossRef] [PubMed]
- Bajaj, J.S.; O’Leary, J.G.; Reddy, K.R.; Wong, F.; Biggins, S.W.; Patton, H.; Fallon, M.B.; Garcia-Tsao, G.; Maliakkal, B.; Malik, R.; et al. Survival in infection-related acute-on-chronic liver failure is defined by extrahepatic organ failures. Hepatology 2014, 60, 250–256. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wang, F.; Sun, W.; Xiao, Q.; Liang, C.; Jiang, S.; Lian, Y.; Shao, J.; Tan, S.; Zheng, S. Peripheral T lymphocytes predict the severity and prognosis in patients with HBV-related acute-on-chronic liver failure. Medicine 2021, 100, e24075. [Google Scholar] [CrossRef] [PubMed]
- Belcher, J.M.; Garcia-Tsao, G.; Sanyal, A.J.; Bhogal, H.; Lim, J.K.; Ansari, N.; Coca, S.G.; Parikh, C.R.; Consortium, T.-A. Association of AKI with mortality and complications in hospitalized patients with cirrhosis. Hepatology 2013, 57, 753–762. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Jalan, R.; Pavesi, M.; Saliba, F.; Amoros, A.; Fernandez, J.; Holland-Fischer, P.; Sawhney, R.; Mookerjee, R.; Caraceni, P.; Moreau, R.; et al. The CLIF Consortium Acute Decompensation score (CLIF-C ADs) for prognosis of hospitalised cirrhotic patients without acute-on-chronic liver failure. J. Hepatol. 2015, 62, 831–840. [Google Scholar] [CrossRef] [Green Version]
- Chancharoenthana, W.; Leelahavanichkul, A. Acute kidney injury spectrum in patients with chronic liver disease: Where do we stand? World J. Gastroenterol 2019, 25, 3684–3703. [Google Scholar] [CrossRef]
- Adebayo, D.; Morabito, V.; Davenport, A.; Jalan, R. Renal dysfunction in cirrhosis is not just a vasomotor nephropathy. Kidney Int. 2015, 87, 509–515. [Google Scholar] [CrossRef] [Green Version]
- Muslimovic, A.; Rasic, S.; Tulumovic, D.; Hasanspahic, S.; Rebic, D. Inflammatory Markers and Procoagulants in Chronic Renal Disease Stages 1–4. Med. Arch. 2015, 69, 307–310. [Google Scholar] [CrossRef] [Green Version]
- Gines, P.; Sola, E.; Angeli, P.; Wong, F.; Nadim, M.K.; Kamath, P.S. Hepatorenal syndrome. Nat. Rev. Dis. Prim. 2018, 4, 23. [Google Scholar] [CrossRef]
- Northup, P.G.; Caldwell, S.H. Coagulation in liver disease: A guide for the clinician. Clin. Gastroenterol. Hepatol. 2013, 11, 1064–1074. [Google Scholar] [CrossRef]
- Li, J.; Liang, X.; You, S.; Feng, T.; Zhou, X.; Zhu, B.; Luo, J.; Xin, J.; Jiang, J.; Shi, D.; et al. Development and validation of a new prognostic score for hepatitis B virus-related acute-on-chronic liver failure. J. Hepatol. 2021, 75, 1104–1115. [Google Scholar] [CrossRef] [PubMed]
- Wu, T.; Li, J.; Shao, L.; Xin, J.; Jiang, L.; Zhou, Q.; Shi, D.; Jiang, J.; Sun, S.; Jin, L.; et al. Development of diagnostic criteria and a prognostic score for hepatitis B virus-related acute-on-chronic liver failure. Gut 2018, 67, 2181–2191. [Google Scholar] [CrossRef] [PubMed]
- Sajja, K.C.; Mohan, D.P.; Rockey, D.C. Age and ethnicity in cirrhosis. J. Investig. Med. 2014, 62, 920–926. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1.
Flowchart of the study.
Figure 2.
Relative importance of the variables in the XGBoost model. The chart shows that HDL-C was the most important variable, followed by Cre, WBC, INR, etc. Abbreviations: hdlc, high-density lipoprotein cholesterol; cre2, creatinine; wbc, white blood cell count; lninr, international normalized ratio; tc, total cholesterol; ldlc, low-density lipoprotein cholesterol; pt, prothrombin time; alb, albumin; tbil2, total bilirubin.
Figure 2.
Relative importance of the variables in the XGBoost model. The chart shows that HDL-C was the most important variable, followed by Cre, WBC, INR, etc. Abbreviations: hdlc, high-density lipoprotein cholesterol; cre2, creatinine; wbc, white blood cell count; lninr, international normalized ratio; tc, total cholesterol; ldlc, low-density lipoprotein cholesterol; pt, prothrombin time; alb, albumin; tbil2, total bilirubin.
Figure 3.
SHAP plotting of the XGBoost model. The closer the variable value is to 1.5 at the x-axis, the higher the possibility of mortality at 30-day. Abbreviations: wbc, white blood cell count; hdlc, high-density lipoprotein cholesterol; tc, total cholesterol; cre2, creatinine; pt, prothrombin time; ldlc, low-density lipoprotein cholesterol; na, blood sodium; pre-alb, pre-albumin; alb, albumin; SHAP, SHapley additive explanation.
Figure 3.
SHAP plotting of the XGBoost model. The closer the variable value is to 1.5 at the x-axis, the higher the possibility of mortality at 30-day. Abbreviations: wbc, white blood cell count; hdlc, high-density lipoprotein cholesterol; tc, total cholesterol; cre2, creatinine; pt, prothrombin time; ldlc, low-density lipoprotein cholesterol; na, blood sodium; pre-alb, pre-albumin; alb, albumin; SHAP, SHapley additive explanation.
Figure 4.
PDP for the key variables in the XGBoost model. The chart shows that creatinine, white blood cell, international normalized ratio, and age showed upward trends with the mortality, while high-density lipoprotein cholesterol and total cholesterol showed opposite trends. Abbreviations: PDP, partial dependence plot; HDL-C, high-density lipoprotein cholesterol; CRE, creatinine; WBC, white blood cell count, INR, international normalized ratio; TC, total cholesterol.
Figure 4.
PDP for the key variables in the XGBoost model. The chart shows that creatinine, white blood cell, international normalized ratio, and age showed upward trends with the mortality, while high-density lipoprotein cholesterol and total cholesterol showed opposite trends. Abbreviations: PDP, partial dependence plot; HDL-C, high-density lipoprotein cholesterol; CRE, creatinine; WBC, white blood cell count, INR, international normalized ratio; TC, total cholesterol.
Figure 5.
LIME of the XGBoost model in four cases from the validation set. The graph of LIME shows how the important variables (in the XGBoost model) contributed to mortality based on four selected samples (two from the survival group, two from the mortality group). Lable p0 means a survival sample, while lable p1 means a dead sample. Abbreviations: LIME, Local Interpretable Model Agnostic Explanation; hdlc, high-density lipoprotein cholesterol; cre2, creatinine; tc, total cholesterol; wbc, white blood cell count.
Figure 5.
LIME of the XGBoost model in four cases from the validation set. The graph of LIME shows how the important variables (in the XGBoost model) contributed to mortality based on four selected samples (two from the survival group, two from the mortality group). Lable p0 means a survival sample, while lable p1 means a dead sample. Abbreviations: LIME, Local Interpretable Model Agnostic Explanation; hdlc, high-density lipoprotein cholesterol; cre2, creatinine; tc, total cholesterol; wbc, white blood cell count.
Table 1.
Characteristics of patients with non-cholestatic cirrhosis by 30-day mortality.
| Total (n = 932) | Survival (n = 796) | Mortality (n = 136) | p | |
|---|---|---|---|---|
| Sex, n (%) | 0.742 | |||
| Male | 581(62) | 494(62) | 87(64) | |
| Female | 351(38) | 302(38) | 49(36) | |
| Age (years) | 61(50, 69) | 60(49, 69) | 67(53, 73) | <0.001 |
| BMI | 23.24 ± 3.19 | 23.33 ± 3.14 | 22.50 ± 3.56 | 0.074 |
| Etiology | 0.784 | |||
| ALD | 80(9) | 67(8) | 13(10) | |
| Others | 852(91) | 729(92) | 123(90) | |
| Complication, n (%) | <0.001 | |||
| Ascites | 123(13) | 115(14) | 8(6) | |
| Hypersplenism | 69(7) | 67(8) | 2(1) | |
| EGVB | 283(30) | 247(31) | 36(26) | |
| HE | 102(11) | 79(10) | 23(17) | |
| Hypohepatia | 355(38) | 288(37) | 67(49) | |
| WBC (1012 g/L) | 4.87(3.30, 7.23) | 4.53(3.15, 6.48) | 7.29(4.91, 10.91) | <0.001 |
| PLT (109 g/L) | 77.00(50.00, 124.25) | 78.00(51.00, 126.00) | 76.00(49.00, 112.25) | 0.489 |
| TBIL (μmol/L) | 2.14(1.23, 4.46) | 1.99(1.20, 3.75) | 4.93(1.93, 13.59) | <0.001 |
| Creatinine (mg/dL) | 0.74(0.60, 0.94) | 0.73(0.59, 0.92) | 0.84(0.65, 1.40) | <0.001 |
| ALT (U/L) | 28.00(18.00, 52.02) | 27.55(17.90, 48.00) | 33.10(19.67, 89.50) | 0.003 |
| AST (U/L) | 41.65(28.67, 71.75) | 40.00(28.08, 67.53) | 52.95(32.88, 106.40) | <0.001 |
| Albumin (g/L) | 30.40(26.67, 34.10) | 30.70(27.00, 34.70) | 28.00(23.98, 31.50) | <0.001 |
| Na (mmol/L) | 139.50(137.00, 141.80) | 139.50(137.00, 141.80) | 138.30(134.67, 141.27) | 0.01 |
| TC (mmol/L) | 3.14(2.50, 3.91) | 3.21(2.55, 4.00) | 2.64(2.00, 3.17) | <0.001 |
| TG (mmol/L) | 0.88(0.66, 1.24) | 0.88(0.66, 1.22) | 0.94(0.64, 1.30) | 0.4 |
| HDL-C (mg/dL) | 0.87(0.52, 1.17) | 0.90(0.62, 1.21) | 0.47(0.21, 0.78) | <0.001 |
| LDL-C (mg/dL) | 1.60(1.18, 2.16) | 1.65(1.26, 2.20) | 1.21(0.89, 1.85) | <0.001 |
| Pre-ALB (mg/dL) | 73.35(48.00, 104.65) | 76.55(50.98, 108.20) | 58.90(39.93, 83.30) | <0.001 |
| PT (s) | 15.50(13.70, 17.90) | 15.20(13.50, 17.30) | 18.30(15.60, 22.02) | <0.001 |
| LnINR | 0.30(0.17, 0.45) | 0.28(0.17, 0.42) | 0.48(0.30, 0.68) | <0.001 |
| MELD | 9.89(6.16, 14.83) | 9.41(5.77, 13.37) | 17.11(9.87, 23.73) | <0.001 |
| MELD-Na | 10.43(5.54, 16.46) | 9.62(5.10, 14.94) | 18.07(10.30, 25.39) | <0.001 |
| ALBI | −1.53 ± 0.66 | −1.59 ± 0.64 | −1.14 ± 0.62 | <0.001 |
Abbreviations: ALBI, albumin-bilirubin; ALD, alcoholic liver disease; ALT, alanine transaminase; AST, aspartate aminotransferase; EGVB, esophagogastric variceal bleeding; HDL-C, high-density lipoprotein cholesterol; HE, hepatic encephalopathy; LDL-C, low-density lipoprotein cholesterol; LnINR, Ln(international normalized ratio); MELD, a model for end-stage liver disease; MELD-Na, MELD combined serum sodium concentration; PLT, platelet count; Pre-ALB, prealbumin; PT, prothrombin time; TBIL, total bilirubin; TC, cholesterol; TG, triglyceride; WBC, white blood cell.
Table 2.
Performance of the AutoML models and the existing scoring systems in predicting mortality in patients with non-cholestatic cirrhosis.
Table 2.
Performance of the AutoML models and the existing scoring systems in predicting mortality in patients with non-cholestatic cirrhosis.
| Dataset | Algorithm | AUC | Accuracy | Sensitivity | Specificity |
|---|---|---|---|---|---|
| Training | GBM | 0.900 | 0.876 | 0.597 | 0.926 |
| XGBoost | 0.938 | 0.915 | 0.647 | 0.963 | |
| LR | 0.662 | 0.852 | 0.555 | 0.905 | |
| RF | 0.807 | 0.833 | 0.513 | 0.890 | |
| DL | 1.000 | 0.999 | 1.000 | 0.999 | |
| Stacking | 0.956 | 0.922 | 0.655 | 0.969 | |
| MELD | 0.806 | 0.784 | 0.736 | 0.793 | |
| MELD-Na | 0.791 | 0.797 | 0.669 | 0.820 | |
| ALBI | 0.739 | 0.649 | 0.752 | 0.630 | |
| Validation | GBM | 0.886 | 0.857 | 0.412 | 0.919 |
| XGBoost | 0.888 | 0.879 | 0.471 | 0.935 | |
| LR | 0.673 | 0.821 | 0.353 | 0.886 | |
| RF | 0.866 | 0.821 | 0.294 | 0.894 | |
| DL | 0.830 | 0.850 | 0.471 | 0.902 | |
| Stacking | 0.850 | 0.871 | 0.294 | 0.951 | |
| MELD | 0.778 | 0.864 | 0.588 | 0.902 | |
| MELD-Na | 0.782 | 0.857 | 0.588 | 0.894 | |
| ALBI | 0.662 | 0.536 | 0.824 | 0.496 |
Abbreviations: DL, deep learning; GBM, gradient boost machine; LR, logistic regression; RF, random forest; XGBoost, eXtreme gradient boosting.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Yu, C.; Li, Y.; Yin, M.; Gao, J.; Xi, L.; Lin, J.; Liu, L.; Zhang, H.; Wu, A.; Xu, C.; et al. Automated Machine Learning in Predicting 30-Day Mortality in Patients with Non-Cholestatic Cirrhosis. J. Pers. Med. 2022, 12, 1930. https://doi.org/10.3390/jpm12111930
AMA Style
Yu C, Li Y, Yin M, Gao J, Xi L, Lin J, Liu L, Zhang H, Wu A, Xu C, et al. Automated Machine Learning in Predicting 30-Day Mortality in Patients with Non-Cholestatic Cirrhosis. Journal of Personalized Medicine. 2022; 12(11):1930. https://doi.org/10.3390/jpm12111930
Chicago/Turabian StyleYu, Chenyan, Yao Li, Minyue Yin, Jingwen Gao, Liting Xi, Jiaxi Lin, Lu Liu, Huixian Zhang, Airong Wu, Chunfang Xu, and et al. 2022. "Automated Machine Learning in Predicting 30-Day Mortality in Patients with Non-Cholestatic Cirrhosis" Journal of Personalized Medicine 12, no. 11: 1930. https://doi.org/10.3390/jpm12111930
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
