Next Article in Journal
Explanatory Models of Burnout Diagnosis Based on Personality Factors and Depression in Managing Nurses
Previous Article in Journal
Transarterial Chemoembolization for Hepatocellular Carcinoma: Why, When, How?
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Machine Learning Model-Based Simple Clinical Information to Predict Decreased Left Atrial Appendage Flow Velocity

1
Chinese PLA Medical School, Haidian District, Beijing 100039, China
2
Chinese PLA General Hospital, Haidian District, Beijing 100039, China
3
School of Medicine, Nankai University, Tianjin 300071, China
4
Department of Cardiology, The Sixth Medical Center of PLA General Hospital, Haidian District, Beijing 100039, China
*
Author to whom correspondence should be addressed.
J. Pers. Med. 2022, 12(3), 437; https://doi.org/10.3390/jpm12030437
Submission received: 11 February 2022 / Revised: 3 March 2022 / Accepted: 7 March 2022 / Published: 10 March 2022
(This article belongs to the Topic Complex Systems and Artificial Intelligence)

Abstract

:
Background: Transesophageal echocardiography (TEE) is the first technique of choice for evaluating the left atrial appendage flow velocity (LAAV) in clinical practice, which may cause some complications. Therefore, clinicians require a simple applicable method to screen patients with decreased LAAV. Therefore, we investigated the feasibility and accuracy of a machine learning (ML) model to predict LAAV. Method: The analysis included patients with atrial fibrillation who visited the general hospital of PLA and underwent transesophageal echocardiography (TEE) between January 2017 and December 2020. Three machine learning algorithms were used to predict LAAV. The area under the receiver operating characteristic curve (AUC) was measured to evaluate diagnostic accuracy. Results: Of the 1039 subjects, 125 patients (12%) were determined as having decreased LAAV (LAAV < 25 cm/s). Patients with decreased LAAV were fatter and showed a higher prevalence of persistent AF, heart failure, hypertension, diabetes and stroke, and the decreased LAAV group had a larger left atrium diameter and a higher serum level of NT-pro BNP than the control group (p < 0.05). Three machine-learning models (SVM model, RF model, and KNN model) were developed to predict LAAV. In the test data, the RF model performs best (R = 0.608, AUC = 0.89) among the three models. A fivefold cross-validation scheme further verified the predictive ability of the RF model. In the RF model, NT-proBNP was the factor with the strongest impact. Conclusions: A machine learning model (Random Forest model)-based simple clinical information showed good performance in predicting LAAV. The tool for the screening of decreased LAAV patients may be very helpful in the risk classification of patients with a high risk of LAA thrombosis.

1. Background

Atrial fibrillation is the most common arrhythmia in clinical practice and is associated with stroke [1]. As shown in previous reports, the left atrial appendage (LAA) may harbor up to 90% of thrombi occurring in patients with AF [2].
The left atrial appendage flow velocity (LAAV) can reflect left atrial appendage function, which has many clinical implications. First, a low LAAV indicates a high risk of LAA spontaneous echo contrast (SEC) and LAA thrombus [3], and SEC is an independent risk factor for subsequent thromboembolic events [4]. Second, LAAV was proved to be an independent predictor of cardioversion success [5]. The measurement of LAAV could provide useful information for the prediction of cardioversion outcomes in AF patients. Third, previous studies have proved that a low LAAV was associated with AF recurrence after the initial catheter ablation of persistent AF [5]. Moreover, low flow velocity in LAA is a predisposing factor in AF patients [6].
Now, transesophageal echocardiography (TEE) is the first technique of choice to evaluate the LAAV [7]. Although TEE is relatively safe and noninvasive, the insertion and manipulation of the ultrasound probe may cause some complications such as oropharyngeal, esophageal, or gastric trauma [8,9]. Therefore, clinicians require a simple applicable method with high sensitivity, which could screen patients with decreased LAAV.
Machine learning (ML) has been used and has shown an acceptable performance in predicting the risk of diseases [10,11,12]. Therefore, in the current study, we used simple clinical data obtained by an ML model to predict LAAV and aim to examine the feature importance of the ML model to understand its mechanism.

2. Methods

2.1. Participants

In the present study, we have retrospectively recruited patients who visiting the General Hospital of PLA and underwent transesophageal echocardiography in between January 2017 and December 2020. Inclusion criteria: (1) age ≥ 18 years old; (2) data documented in HIS system in hospital; (3) complete transesophageal echocardiography examination; (4) diagnosed with nonvalvular atrial fibrillation. Subjects were excluded if they did not meet the above criteria.

2.2. Transesophageal Echocardiography

Left atrial appendage flow velocity was measured by TEE, and all transesophageal echocardiography procedures were performed by experienced cardiologists. Evaluation of LAA by TEE included the presentation or absence of left atrial thrombus, left atrial appendage spontaneous echo contrast, and decreased left atrial appendage emptying peak flow velocity. Based on previous research, LAAV of <25 cm/s was suggested as a useful cutoff value to discriminate patients with a high risk of systemic embolization [13,14].

2.3. Clinical Factors

All clinical data were searched for by the HIS system (Hospital Information System). Since the model we built is a primary screening model, covariates for the machine leaning model were chosen based on prior literature review and clinical judgment, focusing on easily accessible variables that were expected to affect the LAAV.
Predictors in our model included demographics (age and sex), previous medical history (history of atrial fibrillation, hypertension, heart failure, diabetes, stroke, and vascular diseases), anticoagulant drugs and left atrial (LA) diameter. In addition, many studies have proved the correlation between NT-proBNP and LAA flow velocity [15,16,17]. Therefore, we chose BNP as one of the predictors.
To ensure accuracy, all the data were collected twice. If there was any inconsistency, a third review was conducted.

2.4. Study Design

As shown in Figure 1, the data were randomly split into a training set (80%) and a test set (20%). The CreateDataPartition function (caret package) in the R 15.6 Available online: http://www.R-project.org (accessed on 11 February 2022). was used to segment the dataset, and a statistical test was carried out to ensure the balance of the variables in the two datasets. A training set was used for the training of the ML model, and the models were tested in the testing set by cross-validation and 5-fold validation.

2.5. Model Development

In the current study, we built regression models using various ML algorithms, including the support vector machine (SVM) model, random forest (RF) model, and k-nearest neighbor (KNN) model because they are used widely in regression models [18,19,20,21]. All models were developed by R 15.6.

2.6. Model Evaluation

The optimal cut-off point of the predicted LAAV was determined by receiver operating characteristic curve (based on the principle of maximum Yordon index), using LAAV measured by TEE as a gold standard (<25 cm/s). With the cut-off values, the sensitivity, specificity, and F1 score of three models were calculated. Accuracy, sensitivity, specificity, mean squared error and area under the receiver operating characteristic curve (AUCROC), and F1 score were used to evaluate the performance of the model. Accuracy refers to the ratio of the number of correctly predicted decreased LAAV to the total number of participants. Among the metrics, AUCROC and F1 score were the main metrics to reflect the performance of the model.

2.7. Fivefold Cross-Validation

Considering that the model may suffer from overfitting, we performed a fivefold cross-validation scheme. First, the data were partitioned into 5 equal parts. The model was trained on 4 parts, leaving 1 part for testing. The process was repeated 5 times until testing was performed on all of the 5 parts.

3. Definition and Statistics of Variables

Continuous variables (age, Body Mass Index (BMI), serum of NT-proBNP and left atrial diameter) are presented as medians (interquartile ranges) and compared by Mann–Whitney U tests. Categorical variables (gender, anticoagulant, history of persistent AF, heart failure, hypertension, diabetes, stroke, and vascular disease) were presented as number (proportion) and compared by Pearson’s χ2 test or Fisher’s exact test. A p-value of less than 0.05 was considered significant.

4. Results

4.1. Patient Characteristics

Of the 1152 patients undergoing TEE, 1039 patients were included in the final analysis (Figure 1). Among the 1039 patients, 125 patients (12%) were diagnosed with decreased LAAV (<25 cm/s).
As shown in Table 1, patients with low LAAV flow showed a higher prevalence of persistent AF (66.4% vs. 27.7%, p < 0.001), heart failure (32.8 vs. 7.3%, p < 0.001), hypertension (63.2% vs. 51.9%, p = 0.011), diabetes (31.2% vs. 21.9%, p = 0.015), and stroke (21.6% vs. 13.9%, p = 0.02). Left atrial diameter (LA diameter) was larger in patients with decreased LAAV than in those with normal LAAV (35–43 vs. 35–42, p < 0.001), and patients with decreased LAAV were fatter than those in the control group (BMI 26.9 vs. 25.8, p = 0.01). The serum level of NT-pro BNP was higher in the decreased-LAAV group than in the normal-LAAV group (Table 1).

4.2. The Characteristics of the Training Set and Testing Set

As shown in Table 2, of the 1039 patients, 834 patients (80%) were included in the training set, and the remaining 259 patients (20%) were included in the test set. The ratio of decreased LAAV was not significantly different in the training set compared to the test set (11.6% vs. 13.7%, p = 0.424). Patients in the training set showed a similar prevalence of persistent AF (31.7% vs. 35.1%, p = 0.342), heart failure (9.8% vs. 12.7%, p = 0.231), hypertension (53.8% vs. 50.7%, p = 0.425), diabetes (22.9% vs. 23.4%, p = 0.876), stroke (14.9% vs. 14.6%, p = 0.933), and vascular disease (9.4% vs. 6.8%, p = 0.255). The LAAV was not significantly different in the training set than in the testing set (p = 0.727).

4.3. Development of Machine Learning Model

The development of the ML models is shown in Figure 2. In the KNN model, the K value was adjusted to develop the best model, and the performance of the KNN model suggested the best accuracy with a K value of 49 (the absolute error: 13.4 cm/s). The development of the RF model and the KNN model is shown in Figure 2. In the RF model, the number of trees was 500. The support vector machine (SVM) model was also developed based on the training dataset.

4.4. Model Comparison for Regression and Binary Classification Problem

The ability of the three ML models to predict the LAAV is shown in Table 3 (training set) and Table 4 (testing set). The RF model performed the best of the three models. In the testing set, the root-mean-square errors (RMSEs) of the three models were 17.51 cm/s (KNN model), 16.65 cm/s (RF model) and 17.66 cm/s (SVM model), and the mean absolute error of the RF model was also the smallest among the three models (13.4 cm/s, 13.04 cm/s and 13.68 cm/s, respectively).
The ability of the ML algorithms to discriminate between decreased LAAV and normal LAAV is shown in Table 3 and Table 4.
In the training set, the KNN model showed the poorest performance, with an AUCROC of 0.81 (0.76–0.85), while the RF model had the highest AUCROC of 0.98 (0.97–0.99) (RF model vs. KNN model, p < 0.001; RF model vs. SVM model, p < 0.001). The cut-off values were added in Table 3. With the cut-off values, the sensitivity, specificity, accuracy, and F1 score of the three models were calculated. The RF model has the best accuracy (92%) and highest F1 score among three models (KNN 0.44; RF 0.77; SVM 0.58).
In the testing set, the KNN model showed the poorest performance, with an AUCROC of 0.81 (0.73–0.89), while there was no difference found in AUCROC between the RF model and SVM model (p = 0.373). As shown in Table 4, although the sensitivity of the RF model (81%) was lower than the KNN model (93.5%) and SVM model (100%), the specificity of the RF model was highest (RF 86%, KNN 58% and SVM 62%) with the highest accuracy of 85% and highest F1 score of 0.62 (KNN 0.43, SVM 0.48).
The calibration plots of the models were shown in Figure 3. A Hosmer–Lemeshow test was used to evaluate the classification efficiency of the three models, and p-values were shown in Figure 3. All models indicated appropriate calibration (p > 0.05).
As reported in previous studies, a model with AUC ranging from a range of 0.80 to 0.90 is considered excellent [22]. The RF model showed acceptable AUC in both training and testing sets (0.98 and 0.89) and performed better than other non-invasive methods using CT [14,23].
The results of the fivefold cross-validation showed that the RF model showed a better discriminative ability for decreased LAAV than the other two models (Table 5).

4.5. Factors Predicting Decreased LAAV in the RF Model

The overall attributions of variables in the Random Forest model are shown in Figure 4.
The percentages of increase in mean square error (%IncMSE) and increased node purity (IncNodePurity) were used to evaluate the importance of each variable in the model. The serum level of NT-pro BNP contributed the most (%IncMSE: 34.5) to the LAAV prediction, followed by the diagnosis of persistent AF (%IncMSE: 28.0), LA size (%IncMSE: 25.6%), BMI (%IncMSE: 14.7), weight (%IncMSE: 13.3%), etc. IncNodePurity relates to the loss function, which is chosen by best splits. The loss function is MSE for regression and Gini-impurity for classification. In the RF model, the serum level of NT-pro BNP, LA size, BMI, and diagnosis of persistent AF played an important role with high IncNodePurity in all variables.

5. Discussion

We developed three machine learning models (KNN model, Random Forest model and SVM model) to predict LAAV in AF patients using simple clinical risk factors. The models were trained using data from 834 patients and tested using data from 205 patients. In this retrospective analysis, the RF model demonstrated the highest accuracy (AUCROC: 0.89, MAE: 13.04 cm/s) of the three models when validating in the testing set. In the RF model, the serum level of NT-pro BNP contributed the most to the LAAV prediction, followed by LA size, diagnosis of persistent AF, BMI, weight, etc.
The left atrial appendage was proved to be a major source of emboli responsible for cardioembolic stroke in previous studies [24], and decreased LAAV has been proved to be a well-recognized risk factor for left atrial appendage thrombosis and stroke [6,24,25,26]. In clinical practice, TEE was regarded as the first technique of choice to measure the function of left atrial appendage, but TEE may cause severe discomfort in patients and, moreover, serious complications (such as esophageal damage) [27]. Freitas-Ferraz et al. conducted a study including 1249 consecutive patients undergoing TEE and found that the overall incidence of TEE-related complications was 0.9% to 6.1% [28]. Therefore, clinicians require a noninvasive method to screen patients with decreased LAAV.
Coletta et al. [23] proved that transthoracic echocardiography (TTE) could be used to identify patients with low and high blood flow velocities, but only 84% of the patients could be measured by TTE, and this study was conducted with a small number of patients (86 patients). Yasuoka et al. [14] found another noninvasive method to predict LAAV using enhanced computed tomography. They found that the LAAV could be estimated by the HU density ratio at distal and proximal sites within the LAA. However, the study was conducted with a small number of patients (60 patients) and, in clinical practice, the HU density ratio of many patients could not always be measured because of poor-contrast filling of the left atrial appendage. In this study, using noninvasive clinical data, we developed an ML model to predict the LAAV of AF patients, which could be used to primarily screen patients with decreased LAAV in a cheap and fast way.
Machine learning has shown acceptable results in various medical fields. In ML model development, to avoid overfitting of the model, it has been suggested that the number-to-feature ratio should be at least 5 [29]. In the present study, the ratio was about 70. The data were randomly divided into two subsets, and the features in the testing set were not significantly different than the training set. In cross-validation, the RF model had 86% specificity and 81% sensitivity. We further assessed the model by fivefold cross-validation, and, consistently with the previous results of cross-validation, the RF model performed best (AUVROC 0.85).
Goldman et al. found that age, systolic blood pressure, sustained AF, ischemic heart disease, and left atrial area were associated with LAAV [30]. Demircelik et al. found that left ventricular diastolic dysfunction was associated with left atrial appendage functions [31]. Handke et al. found that left ventricular ejection fraction, LA size, paroxysmal AF, age, and sex are independent parameters influencing LAAV [32]. Although many clinical factors associated with decreased LAAV are increasingly available, risk estimation of decreased LAAV remains challenging. The indicators included in our model were easily accessible; therefore, our model may enable instantaneous risk estimation of decreased LAAV, which may facilitate rapid identification of individuals at elevated risk to guide further invasive inspection.
Although machine learning models may be useful to help us to diagnose patients, they are still a “black box”, lacking acceptable interpretability. In the RF model, we further uncovered important predictors of decreased LAAV. Based on our results, serum level of NT-proBNP and LA size was the most important variable (Figure 4), which was also confirmed as a risk factor in previous research [16,33]. The percentage of IncMSE (% increased mean square error) was equivalent to mean decrease accuracy, which could be used to measure the importance of each variable in the model. Other factors, such as diagnosis of persistent AF, weight, and BMI, were still proven to be useful for predicting LAAV. Because machine learning models can quickly process large amounts of data, we note other implications of our study, which suggest that machine learning models may contribute to extract elements of risk.
It is worth noting the use of anticoagulants in AF patients. A previous study has shown that the underuse of OAC is common [34]. In 2008, only fewer than half were treated with anticoagulation (2.7%, warfarin; 39.7%, aspirin) [35]. In 2016, Xiong et al. found that only 44.5% of Chinese AF patients received OAC treatment in a research-based Chinese population [36]. In recent years, the use of anticoagulants has gradually increased but remains inadequate. In our results, about half of the AF patients (50.8%) were not taking anticoagulants, which indicates that standardized anticoagulant use remains inadequate.
In our model, the serum level of NT-proBNP was one of the most important variables. Several mechanisms may explain the association between NT-proBNP and decreased LAAV. BNP is secreted mainly from the left atrium, and atrial pressure overload leads to elevation in plasma BNP levels [37]. Previous studies [38,39] found that elevated BNP was associated with LAA disfunction, and a high plasma brain natriuretic polypeptide level was a marker of risk for thromboembolism in patients with nonvalvular atrial fibrillation [40]. Harada et al. also found that higher plasma BNP was associated with a lower LAA flow velocity in patients with nonvalvular AF and normal LV systolic function [17]. Therefore, in addition to diagnosing heart failure, the serum level of NT-pro BNP could, to a certain degree, reflect the function of left atrium and left atrial appendage. Therefore, it plays a great role in predicting the blood flow velocity of the left atrial appendage.

6. Limitations

This study has some limitations. First, this study was a single-center and retrospective study, and had a limitation in generalizability. Second, the model was only tested in the test set instead of a prospective cohort in the real world. Third, there may be some missing clinical features in the candidate features which may contribute to the improvement of the model.

7. Conclusions

Machine learning model (Random Forest model)-based simple clinical information showed good performance in predicting LAAV. The tool for screening decreased LAAV patients may be very helpful in the risk classification of patients with a high risk of LAA thrombosis.

Author Contributions

Supervision, J.Y.; writing—original draft, C.L.; writing—review and editing, C.L., Y.C., J.G., and J.Y.; data curation—G.D., Y.D., R.X., and J.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by grants from the National Key R&D Program of China (2016YFC1300304 and 2021YFC2500505), and Medical Big Data Program of PLAGH (2019MBD-035).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee of General Hospital of PLA (Project identification code: CardioDB2009-2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy issues.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Schnabel, R.B.; Yin, X.; Gona, P.; Larson, M.G.; Beiser, A.; McManus, D.D.; Newton-Cheh, C.; Lubitz, S.A.; Magnani, J.W.; Ellinor, P.; et al. 50-year trends in atrial fibrillation prevalence, incidence, risk factors, and mortality in the Framingham Heart Study: A cohort study. Lancet 2015, 386, 154–162. [Google Scholar] [CrossRef] [Green Version]
  2. Blackshear, J.L.; Odell, J.A. Appendage obliteration to reduce stroke in cardiac surgical patients with atrial fibrillation. Ann. Thorac. Surg. 1996, 61, 755–759. [Google Scholar] [CrossRef]
  3. Pollick, C.; Taylor, D. Assessment of left atrial appendage function by transesophageal echocardiography. Implications for the development of thrombus. Circulation 1991, 84, 223–231. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Akamatsu, K.; Ito, T.; Ozeki, M.; Miyamura, M.; Sohmiya, K.; Hoshiga, M. Left atrial spontaneous echo contrast occurring in patients with low CHADS2 or CHA2DS2-VASc scores. Cardiovasc. Ultrasound 2020, 18, 31. [Google Scholar] [CrossRef] [PubMed]
  5. Kanda, T.; Masuda, M.; Sunaga, A.; Fujita, M.; Iida, O.; Okamoto, S.; Ishihara, T.; Watanabe, T.; Takahara, M.; Sakata, Y.; et al. Low left atrial appendage flow velocity predicts recurrence of atrial fibrillation after catheter ablation of persistent atrial fibrillation. J. Cardiol. 2015, 66, 377–381. [Google Scholar] [CrossRef] [Green Version]
  6. Takada, T.; Yasaka, M.; Nagatsuka, K.; Minematsu, K.; Yamaguchi, T. Blood flow in the left atrial appendage and embolic stroke in nonvalvular atrial fibrillation. Eur. Neurol. 2001, 46, 148–152. [Google Scholar] [CrossRef] [PubMed]
  7. Shanewise, J.S.; Cheung, A.T.; Aronson, S.; Stewart, W.J.; Weiss, R.L.; Mark, J.B.; Savage, R.M.; Sears-Rogan, P.; Mathew, J.P.; Quiñones, M.A.; et al. ASE/SCA guidelines for performing a comprehensive intraoperative multiplane transesophageal echocardiography examination: Recommendations of the American Society of Echocardiography Council for Intraoperative Echocardiography and the Society of Cardiovascular Anesthesiologists Task Force for Certification in Perioperative Transesophageal Echocardiography. J. Am. Soc. Echocardiogr. 1999, 12, 884–900. [Google Scholar]
  8. Côté, G.; Denault, A. Transesophageal echocardiography-related complications. Can. J. Anaesth. 2008, 55, 622–647. [Google Scholar] [CrossRef] [Green Version]
  9. Hilberath, J.N.; Oakes, D.A.; Shernan, S.K.; Bulwer, B.E.; D’Ambra, M.N.; Eltzschig, H.K. Safety of transesophageal echocardiography. J. Am. Soc. Echocardiogr. 2010, 23, 1115–1127. [Google Scholar] [CrossRef]
  10. Fan, J.; Chen, M.; Luo, J.; Yang, S.; Shi, J.; Yao, Q.; Zhang, X.; Du, S.; Qu, H.; Cheng, Y.; et al. The prediction of asymptomatic carotid atherosclerosis with electronic health records: A comparative study of six machine learning models. BMC Med. Inform. Decis. Mak. 2021, 21, 115. [Google Scholar] [CrossRef]
  11. Atta-Fosu, T.; LaBarbera, M.; Ghose, S.; Schoenhagen, P.; Saliba, W.; Tchou, P.J.; Lindsay, B.D.; Desai, M.Y.; Kwon, D.; Chung, M.K.; et al. A new machine learning approach for predicting likelihood of recurrence following ablation for atrial fibrillation from CT. BMC Med. Imaging 2021, 21, 45. [Google Scholar] [CrossRef]
  12. Liu, L.; Tan, S.; Li, Y.; Luo, J.; Zhang, W.; Li, S. An early aortic dissection screening model and applied research based on ensemble learning. Ann. Transl. Med. 2020, 8, 1578. [Google Scholar] [CrossRef] [PubMed]
  13. Mügge, A.; Kühn, H.; Nikutta, P.; Grote, J.; Lopez, J.G.; Daniel, W.G. Assessment of left atrial appendage function by biplane transesophageal echocardiography in patients with nonrheumatic atrial fibrillation: Identification of a subgroup of patients at increased embolic risk. J. Am. Coll. Cardiol. 1994, 23, 599–607. [Google Scholar] [CrossRef] [Green Version]
  14. Yasuoka, R.; Kurita, T.; Kotake, Y.; Akaiwa, Y.; Hashiguchi, N.; Motoki, K.; Kobuke, K.; Iwanaga, Y.; Miyazaki, S.; Yamamoto, H.; et al. A novel method to estimate blood flow velocity in the left atrial appendage using enhanced computed tomography: Role of Hounsfield unit density ratio at two distinct points within the left atrial appendage. Hear. Vessel. 2017, 32, 893–901. [Google Scholar] [CrossRef]
  15. Igarashi, Y.; Kashimura, K.; Makiyama, Y.; Sato, T.; Ojima, K.; Aizawa, Y. Left atrial appendage dysfunction in chronic nonvalvular atrial fibrillation is significantly associated with an elevated level of brain natriuretic peptide and a prothrombotic state. Jpn. Circ. J. 2001, 65, 788–792. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Fukuhara, E.; Mine, T.; Kishima, H.; Ishihara, M. Predictors for reduced flow velocity in left atrial appendage during sinus rhythm in patients with atrial fibrillation. Heart Vessel. 2020, 36, 393–400. [Google Scholar] [CrossRef] [PubMed]
  17. Harada, M.; Tabako, S.; Fujii, Y.; Takarada, Y.; Hayashi, K.; Ohara, H.; Hara, F.; Ikeda, T. Correlation between plasma brain natriuretic peptide levels and left atrial appendage flow velocity in patients with non-valvular atrial fibrillation and normal left ventricular systolic function. J. Echocardiogr. 2018, 16, 72–80. [Google Scholar] [CrossRef]
  18. Rahmati, O.; Choubin, B.; Fathabadi, A.; Coulon, F.; Soltani, E.; Shahabi, H.; Mollaefar, E.; Tiefenbacher, J.; Cipullo, S.; Bin Ahmad, B.; et al. Predicting uncertainty of machine learning models for modelling nitrate pollution of groundwater using quantile regression and UNEEC methods. Sci. Total Environ. 2019, 688, 855–866. [Google Scholar] [CrossRef]
  19. Bui, X.-N.; Jaroonpattanapong, P.; Nguyen, H.; Tran, Q.-H.; Long, N.Q. A novel hybrid model for predicting blast-induced ground vibration based on k-nearest neighbors and particle swarm optimization. Sci. Rep. 2019, 9, 13971. [Google Scholar] [CrossRef] [Green Version]
  20. Battineni, G.; Sagaro, G.G.; Chinatalapudi, N.; Amenta, F. Applications of machine learning predictive models in the chronic disease diagnosis. J. Pers. Med. 2020, 10, 21. [Google Scholar] [CrossRef] [Green Version]
  21. Naemi, A.; Schmidt, T.; Mansourvar, M.; Naghavi-Behzad, M.; Ebrahimi, A.; Wiil, U.K. Machine learning techniques for mortality prediction in emergency departments: A systematic review. BMJ Open 2021, 11, e052663. [Google Scholar] [CrossRef]
  22. Lee, Y.-H.; Bang, H.; Kim, D.J. How to establish clinical prediction models. Endocrinol. Metab. 2016, 31, 38–44. [Google Scholar] [CrossRef] [PubMed]
  23. Coletta, C.; Infusino, T.; Sciarretta, S.; Sestili, A.; Trambaiolo, P.; Cianfrocca, C.; De Marchis, E.; Auriti, A.; Salustri, A. Transthoracic Doppler echocardiography for the assessment of left atrial appendage size and blood flow velocity. A multicentre study. J. Cardiovasc. Med. 2008, 9, 147–152. [Google Scholar] [CrossRef]
  24. Lee, Y.; Park, H.C.; Lee, Y.; Kim, S.G. Comparison of morphologic features and flow velocity of the left atrial appendage among patients with atrial fibrillation alone, transient ischemic attack, and cardioembolic stroke. Am. J. Cardiol. 2017, 119, 1596–1604. [Google Scholar] [CrossRef] [PubMed]
  25. Schnieder, M.; Siddiqui, T.; Karch, A.; Bähr, M.; Hasenfuß, G.; Schroeter, M.R.; Liman, J. Low flow in the left atrial appendage assessed by transesophageal echocardiography is associated with increased stroke severity—Results of a single-center cross-sectional study. Int. J. Stroke 2018, 14, 423–429. [Google Scholar] [CrossRef]
  26. Li, Y.-H.; Lai, L.-P.; Shyu, K.-G.; Hwang, J.-J.; Ma, H.-M.; Ko, Y.-L.; Kuan, P.; Lien, W.-P. Clinical implications of left atrial appendage function: Its influence on thrombus formation. Int. J. Cardiol. 1994, 43, 61–66. [Google Scholar] [CrossRef]
  27. Min, J.K.; Spencer, K.T.; Furlong, K.T.; DeCara, J.M.; Sugeng, L.; Ward, R.P.; Lang, R.M. Clinical features of complications from transesophageal echocardiography: A single-center case series of 10,000 consecutive examinations. J. Am. Soc. Echocardiogr. 2005, 18, 925–929. [Google Scholar] [CrossRef] [PubMed]
  28. Freitas-Ferraz, A.B.; Rodés-Cabau, J.; Vega, L.J.; Beaudoin, J.; O’Connor, K.; Turgeon, P.Y.; Paradis, J.-M.; Ferreira-Neto, A.; Asmarats, L.; Champagne, J.; et al. Transesophageal echocardiography complications associated with interventional cardiology procedures. Am. Heart J. 2019, 221, 19–28. [Google Scholar] [CrossRef] [PubMed]
  29. Sengupta, P.P.; Shrestha, S.; Berthon, B.; Messas, E.; Donal, E.; Tison, G.H.; Min, J.K.; D’hooge, J.; Voigt, J.-U.; Dudley, J.; et al. Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council. JACC Cardiovasc. Imaging 2020, 13, 2017–2035. [Google Scholar] [CrossRef]
  30. Goldman, M.E.; Pearce, L.; Hart, R.G.; Zabalgoitia, M.; Asinger, R.W.; Safford, R.; Halperin, J.L. Pathophysiologic correlates of thromboembolism in nonvalvular atrial fibrillation: I. Reduced flow velocity in the left atrial appendage (the stroke prevention in atrial fibrillation [SPAF-III] study). J. Am. Soc. Echocardiogr. 1999, 12, 1080–1087. [Google Scholar] [CrossRef]
  31. DemirCelik, M.B.; Cetin, M.; Cicekcioglu, H.; Ucar, O.; Duran, M. Effect of left ventricular diastolic dysfunction on left atrial appendage function and thrombotic potential in nonvalvular atrial fibrillation. Anadolu Kardiyol Derg. 2014, 14, 256–260. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Handke, M.; Harloff, A.; Hetzel, A.; Olschewski, M.; Bode, C.; Geibel, A. Left atrial appendage flow velocity as a quantitative surrogate parameter for thromboembolic risk: Determinants and relationship to spontaneous echocontrast and thrombus formation–a transesophageal echocardiographic study in 500 patients with cerebral ischemia. J. Am. Soc. Echocardiogr. 2005, 18, 1366–1372. [Google Scholar]
  33. Markl, M.; Lee, D.C.; Furiasse, N.; Carr, M.; Foucar, C.; Ng, J.; Carr, J.; Goldberger, J.J. Left atrial and left atrial appendage 4D blood flow dynamics in atrial fibrillation. Circ. Cardiovasc. Imaging 2016, 9, e004984. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Li, Y.G.; Lee, S.R.; Choi, E.K.; Lip, G.Y. Stroke prevention in atrial fibrillation: Focus on Asian patients. Korean Circ. J. 2018, 48, 665–684. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Zhou, Z.; Hu, D. An epidemiological study on the prevalence of atrial fibrillation in the Chinese population of mainland China. J. Epidemiol. 2008, 18, 209–216. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Xiong, Q.; Shantsila, A.; Li, J.; Hong, K.; Lip, G.Y. Suboptimal oral anticoagulant treatment among Chinese non-valvular atrial fibrillation patients: The Nanchang Atrial Fibrillation Project. Arch. Med. Sci. 2016, 12, 216–218. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Inoue, S.I.; Murakami, Y.O.; Sano, K.; Katoh, H.; Shimada, T. Atrium as a source of brain natriuretic polypeptide in patients with atrial fibrillation. J. Card. Fail. 2000, 6, 92–96. [Google Scholar] [CrossRef]
  38. Kishima, H.; Mine, T.; Fukuhara, E.; Ashida, K.; Ishihara, M. Predictors of left atrial appendage stunning after electrical cardioversion in patients with atrial fibrillation. Int. J. Cardiovasc. Imaging 2019, 35, 1549–1555. [Google Scholar] [CrossRef] [PubMed]
  39. Kurosawa, K.; Negishi, K.; Obokata, M.; Sorimachi, H.; Masuda, K.; Machida, T.; Kurabayashi, M.; Murakami, M. Left atrial strain independently and incrementally predicts high risk thromboembolic findings over CHA2DS2-VASc score and BNP. Rinsho Byori. 2017, 65, 138–146. [Google Scholar]
  40. Shimizu, H.; Murakami, Y.; Inoue, S.I.; Ohta, Y.; Nakamura, K.; Katoh, H.; Sakne, T.; Takahashi, N.; Ohata, S.; Sugamori, T.; et al. High plasma brain natriuretic polypeptide level as a marker of risk for thromboembolism in patients with nonvalvular atrial fibrillation. Stroke 2002, 33, 1005–1010. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Flow diagram. TEE: transesophageal echocardiography; LAAV: left atrial appendage flow velocity.
Figure 1. Flow diagram. TEE: transesophageal echocardiography; LAAV: left atrial appendage flow velocity.
Jpm 12 00437 g001
Figure 2. Development of KNN Model and Random Forest Model. (A) Development of KNN model; (B) Development of Random Forest model.
Figure 2. Development of KNN Model and Random Forest Model. (A) Development of KNN model; (B) Development of Random Forest model.
Jpm 12 00437 g002
Figure 3. Calibration plots for the KNN(A), RF(B), and SVM(C) models. (KNN, k-Nearest Neighbor; RF, random forest; SVM, support vector machine). All models indicated appropriate calibration (p > 0.05). (A) Calibration plot for the KNN model; (B) Calibration plot for the RF model; (C) Calibration plot for the SVM model.
Figure 3. Calibration plots for the KNN(A), RF(B), and SVM(C) models. (KNN, k-Nearest Neighbor; RF, random forest; SVM, support vector machine). All models indicated appropriate calibration (p > 0.05). (A) Calibration plot for the KNN model; (B) Calibration plot for the RF model; (C) Calibration plot for the SVM model.
Jpm 12 00437 g003
Figure 4. The importance of variables in Random Forest model. BMI: body mass index; DM: diabetes mellitus.
Figure 4. The importance of variables in Random Forest model. BMI: body mass index; DM: diabetes mellitus.
Jpm 12 00437 g004
Table 1. Baseline characteristics of the records of the enrolled patients with decreased LAAV and normal LAAV.
Table 1. Baseline characteristics of the records of the enrolled patients with decreased LAAV and normal LAAV.
ALL
n = 1039
Normal LAAV
n = 914
Decreased LAAV
n = 125
p-Value
Age (years)62 (54–69)62 (54–68)64 (54–71.5)0.022
Gender (male)757 (72.9%)663 (72.5%)94 (75.2%)0.530
BMI (kg/m2)26 (23–28)25.8 (23.8–28.0)26.9 (24.4–29.3)0.010
Persistent AF336 (32.3%)253 (27.7%)83 (66.4%)<0.001
Antithrombotic therapy <0.001
No antithrombotic therapy 528 (50.8%)478 (52.3%)50 (40.0%)
Aspirin130 (12.5%)113 (12.4%)17 (13.6%)
Clopidogrel11 (1.1%)8 (0.9%)3 (2.4%)
Dual antiplatelet21 (2.0%)18 (2.0%)3 (2.4%)
Warfarin95 (9.1%)76 (8.3%)19 (15.2%)
NOAC254 (24.5%)221 (24.2%)33 (26.4%)
Heart failure108 (10.4%)67 (7.3%)41 (32.8%)<0.001
Hypertension 553 (53.2%)474 (51.9%)79 (63.2%)0.017
Diabetes 239 (23.0%)200 (21.9%)39 (31.2%)0.020
Stroke 154 (14.8%)127 (13.9%)27 (21.6%)0.023
Vascular disease92 (8.9%)83 (9.1%)9 (7.2%)0.487
NT-pro BNP (pg/mL) 319 (116–770)266 (103–635)997 (590.5–1862)<0.001
LA diameter (mm)39 (35–43)38 (35–42)43 (40–47.5)<0.001
NOAC: New oral anticoagulants, include dabigatran and rivaroxaban; LA diameter: left atrial diameter; BMI: Body Mass Index.
Table 2. The characteristics of training set and testing set.
Table 2. The characteristics of training set and testing set.
Feature Training Set
n = 834
Testing Set
n = 205
p-Value
Age (years)62 (54–69)61 (54–68.5)0.357
Gender(male)592 (71.0%)165 (80.5%)0.006
BMI (kg/m2)26 (23.9–28.1)26 (24.2–28.2)0.772
Persistent AF264 (31.7%)72 (35.1%)0.342
Antithrombotic therapy 0.68
No antithrombotic therapy426 (51.2%)102 (49.8%)
Aspirin104 (12.5%)26 (12.7%)
Clopidogrel10 (1.2%)1 (0.5%)
Dual antiplatelet19 (2.3%)2 (1%)
Warfarin72 (8.6%)23 (11.2%)
NOAC203 (24.3%)51 (24.8%)
Heart failure82 (9.8%)26 (12.7%)0.231
Hypertension 449 (53.8%)104 (50.7%)0.425
Diabetes 191 (22.9%)48 (23.4%)0.876
Stroke 124 (14.9%)30 (14.6%)0.933
Vascular disease78 (9.4%)14 (6.8%)0.255
NT-pro BNP (pg/mL) 309 (111–725)401 (150–885)0.061
LA size (mm)39 (35–43)39 (36–42)0.446
LAAV (cm/s)46 (33–60.25)46 (36–42)0.727
Decreased LAAV97 (11.6%)28 (13.7%)0.424
Table 3. Comparison of the predictive performance for three models in the training set.
Table 3. Comparison of the predictive performance for three models in the training set.
Model Accuracy (%)F1 ScoreAUCCut-off
Value
R2RMSE (cm/s)MAE (cm/s)
KNN760.440.81 (0.76–0.85)41.30.2617.2913.80
RF920.770.98 (0.97–0.99)33.80.857.855.97
SVM840.580.91 (0.87–0.94)37.10.2317.6613.68
KNN, k-nearest neighbor; RF, random forest; SVM, support vector machine; AUC, area under the receiver operating characteristic curve; R2, R Squared; RMSE, root-mean-square error; MAE, mean absolute deviation.
Table 4. Comparison of the predictive performance for three models in the testing set.
Table 4. Comparison of the predictive performance for three models in the testing set.
Model Accuracy
(%)
F1 ScoreAUCSpecificity
(%)
Sensitivity
(%)
R2RMSE (cm/s)MAE (cm/s)
KNN630.430.81 (0.73–0.89)5893.50.2417.5113.40
RF850.620.89 (0.83–0.95)86810.3116.6513.04
SVM670.480.87 (0.82–0.93)621000.2317.6613.68
KNN, k-nearest neighbor; RF, random forest; SVM, support vector machine; AUC, area under the receiver operating characteristic curve; R2, R Squared; RMSE, root-mean-square error; MAE, mean absolute deviation.
Table 5. Comparison of the predictive performance for three models (fivefold cross-validation).
Table 5. Comparison of the predictive performance for three models (fivefold cross-validation).
Model AUCROCR2RMSE (cm/s)MAE (cm/s)
KNN0.8060.2417.4313.40
RF0.8540.73510.287.44
SVM0.840.3915.7611.92
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Li, C.; Dou, G.; Ding, Y.; Xin, R.; Wang, J.; Guo, J.; Chen, Y.; Yang, J. Machine Learning Model-Based Simple Clinical Information to Predict Decreased Left Atrial Appendage Flow Velocity. J. Pers. Med. 2022, 12, 437. https://doi.org/10.3390/jpm12030437

AMA Style

Li C, Dou G, Ding Y, Xin R, Wang J, Guo J, Chen Y, Yang J. Machine Learning Model-Based Simple Clinical Information to Predict Decreased Left Atrial Appendage Flow Velocity. Journal of Personalized Medicine. 2022; 12(3):437. https://doi.org/10.3390/jpm12030437

Chicago/Turabian Style

Li, Chao, Guanhua Dou, Yipu Ding, Ran Xin, Jing Wang, Jun Guo, Yundai Chen, and Junjie Yang. 2022. "Machine Learning Model-Based Simple Clinical Information to Predict Decreased Left Atrial Appendage Flow Velocity" Journal of Personalized Medicine 12, no. 3: 437. https://doi.org/10.3390/jpm12030437

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop