Performance Evaluation and Engineering Verification of Machine Learning Based Prediction Models for Slope Stability
by
Gexue Bai
1, 
Yunlong Hou
1,
Baofeng Wan
1,
Ning An
1,
Yihao Yan
2,
Zheng Tang
2,
Mingchun Yan
2,
Yihan Zhang
2 and
Daoyuan Sun
2,* 1
Gansu Institute of Engineering Geology, Lanzhou 730000, China
2
School of Resources and Safety Engineering, Central South University, Changsha 410083, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(15), 7890; https://doi.org/10.3390/app12157890
Submission received: 8 June 2022
/
Revised: 15 July 2022
/
Accepted: 26 July 2022
/
Published: 6 August 2022
(This article belongs to the Section Earth Sciences)
Abstract
:Featured Application
This study provides a straightforward method to determine the machine learning model with the best predictive performance and demonstrates a complete model building solution for predicting the factor of safety in slope engineering.
Abstract
Stability evaluation of geotechnical engineering slopes is of great significance for the risk control and safe operation of many engineering. Machine learning methods can effectively establish the potential relationship between geological features and slope behavior under complex environments, to accurately evaluate the stability of slope rock and soil. This work investigated the performance of eight commonly used machine learning models to predict slope safety factors. First, the prediction system of slope safety factors based on machine learning was established by combining historical data of slopes for parameter optimization and cross-validation. Then, four accuracy evaluation indexes, MSE, RMSE, MAE, and Pearson correlation, were objectively weighted, and objective weighting-TOPSIS models were constructed to comprehensively quantify the performance of each model. Finally, the best machine learning model was used in the slope stability analysis of the Sino–Russian natural gas control section. The research results show that there are obvious differences in the prediction accuracy of the slope safety factor among different models. The ANN model has the highest evaluation accuracy, and the ensemble learning method performs well in the data set. The machine learning model can better predict the safety factor of the slope under different working conditions. The discrepancies with the numerical simulation results are related to the limitations of data sets and the differences in analysis methods. The analysis method of this study not only provides a new research idea and solution for the construction and evaluation of the model predicting slope safety factors, but also applies to other geotechnical engineering instability problems.
1. Introduction
Slope instability, as one of the most dangerous natural disasters, poses a great hazard to the protection of public property and life. How to evaluate slope stability and give corresponding engineering countermeasures is the top priority of slope engineering research [1]. The limit equilibrium methods commonly used in slope engineering include the Swedish arc method and the simplified Bishop method. Based on the Swedish method, the Bishop method considers the contribution of the force between the strips in the normal direction and calculates the safety factor by the average value of each strip. The overall safety factor is about 10% to 20% higher than that of the Swedish strip method [2,3,4,5,6]. Bishop’s method is often preferred when using the limit equilibrium method to analyze slope stability. For example, Zhang et al. [7] used the ISM method combined with on-site monitoring data to evaluate the slope state at different rainfall depths in real-time. The simplified Bishop method is simple and applicable, and its calculation results are accurate and computerized; therefore, it should be used first in the stability analysis of soil slopes without weak layers. The uncertainty of parameter values makes it difficult for the traditional reliability analysis to correspond with the practical engineering. To solve this problem, Dong et al proposed an innovative method for stability analysis of jointed roclmass combined with interval values and block theory [8], and the microseismic load was further taken into consideration [9]. The interval non-probabilistic reliability analysis method is also applied to the stability analysis of tailings dam [10] and the real-time monitoring and pre-alarm for the disasters of tailings dam in mines [11]. As a common and accurate numerical simulation method for slope stability analysis, the finite element method is the first numerical simulation method applied to soil slope stability analysis [12]. A large number of studies [13,14,15,16,17,18] have combined the finite element strength coefficient reduction method to analyze the slope stability problem, quantitatively revealing the mechanism and process of the progressive failure of the rock and soil mass before and after the instability, and calculating the safety factor of the critical slip surface.
However, traditional numerical methods cannot provide accurate results of slope instability due to the complexity and uncertainty of multiple correlation factors and small unbalanced data samples. Based on the collected geotechnical properties and historical behavior of slope instances, the application of machine learning methods to evaluate slope stability has become an important solution [19,20,21,22,23,24]. Neural networks are widely used in this field, including artificial neural networks [25], back Propagation Neural Networks [26], Differential Evolution Neural Networks [27], and other methods [28]. Li et al. [29] calculated the key sections of the landslide based on 3D modeling and obtained the factor of safety (FOS), which provided a reliable basis for slope stability analysis and comprehensive treatment. Zhou et al. [30] used the GBM method to establish the nonlinear relationship between the safety factor and influencing factors to carry out slope stability analysis. Wei et al. [31] built different support vector regression (SVR) kernels to predict FOS values and the results show that the radius basis function (RBF) kernel could produces more accurate prediction performance. Qi et al. [32] established six integrated artificial intelligence models to predict the slope stability and proved that the integrated AI methods have great potential for slope stability prediction. Machine learning methods provide a new solution for geotechnical slope engineering design and disaster assessment, but few people pay attention to the overall comparison of the performance in different learning models, and it is equally important to determine the best-performed prediction model simply and directly.
This work evaluates 102 slope cases under the arc-shaped instability failure mode using 8 machine learning regression methods. The FOS prediction models of the slope are established by carrying out cross-validation and model hyperparameter adjustment. Further, based on the objective weighting and the TOPSIS methods, a performance evaluation model of the machine learning model is established to determine the best FOS prediction model. Finally, the field data of the slope of the Bei’an-Heihe expressway is used to predict the FOS. The predicted result is verified by the FOS obtained by the Bishop method.
2. Machine Learning Model Development
2.1. Machine Learning Datasets and Feature Parameters
The sample data set is composed of 102 cases of slope failure analysis with circular arc failure mode. The sample features include six parameters related to the geometry and geotechnical characteristics of each slope. Among them, Slope height (h), total slope angle (β), and bulk density (γ) reflect the basic geometric design of the slope. According to the Mohr–Coulomb yield criterion, cohesion (c) and internal friction angle (φ) have an important influence on the stability of the slope. The external trigger factor considered is the pore water ratio (ru), which is the ratio of pore water pressure to overburden pressure. The detailed database for training the machine learning methods are shown in Table 1. Slope stability, as the sample label, is expressed by the safety factor (FOS). If the FOS value is greater than 1.05, the slope is considered stable. It is difficult to analyze the slope stability due to the uncertainty and complexity of the slope. Consistent with the previous slope stability analysis methods, a slope mapping relationship between the safety factor and the characteristic parameters is established through the training process of machine learning.
The value range of each characteristic parameter affecting slope stability is different. For example, the value range of cohesion (c) is 0–150kPa; the value range of internal friction angle (φ) is 0–50°. Further, each sample feature data is standardized, as shown in Figure 1. The comparison of the parameter characteristics of the slope in different states is given in the violin diagram, in which the area represents the relative proportion of the data volume while the dotted lines represent the 15%, 50%, and 75% quantiles of the data in turn. The distribution of ru values is wider, while the distribution of c values is more concentrated, and the φ and β distributions of stable slopes and unstable slopes are similar. The pairwise relationships between all sample data are shown in Figure 2. There is no relatively great or meaningful correlation between the six sample features. There is a certain overlap of characteristic parameters in the distribution of slope stability events and slope instability events, but the characteristic distribution of stability data is generally larger than that of instability (diagonal diagram in Figure 2). The off-diagonal plots in Figure 2 show that the pairwise relationships of characteristic parameters under stable and unstable slopes are clustered in different regions. In summary, each sample feature has an independent role in evaluating slope stability.
2.2. Methods and Hyperparameter Adjustment
In this study, eight commonly used machine learning regression methods, including support vector machine (SVM), decision tree (DT), k-nearest neighbor algorithm (kNN), AdaBoost algorithm (ADA), random forest (RF), artificial neural network (ANN), guided clustering algorithm (Bagging), and gradient boosting decision tree (GBDT), were selected to construct the prediction model of FOS in slope engineering. The data set was segmented by 10-fold cross-validation, and the test set and the training set were circularly and alternately validated. All validation results were averaged to obtain the accuracy of each model. In the training process, the grid search method was used to optimize the parameters of different models and obtain the best model prediction effect as much as possible. Table 2 shows the optimal model of eight machine learning algorithms through cross-validation and parameter settings.
3. Prediction Results
In the process of cross-validation, the original data set is randomly divided into 10 equal-sized subsets using the 10-fold cross-validation method. Among the 10 subsets, one subset is reserved as the verification subset of the test model, and the remaining 9 subsets are used for the training model. Then, this cross-validation process is repeated 10 times, and each of the 10 subsets is only used once as validation data. The Figure 3, Figure 4, Figure 5 and Figure 6 show the prediction result and error in one of the cross-validations. It can be seen that the FOS values calculated by ANN and GBDT are very close to the actual values and the absolute error is less than 0.05, while the test set results are quite different from the actual values in the other six models. In particular, the median and outliers of SVM and kNN errors are significantly higher than other models, which perform poorly in the training set. In the test set, the median GBDT error is the smallest. Meanwhile, the predicted values of FOS obtained by all ensemble learning models are relatively close to the real values, and the errors are generally distributed around 0.15. It should be emphasized that although the calculation result after a certain segmentation of the data set cannot fully objectively reflect the accuracy of the model, it can present a basic judgment in different prediction models.
4. Performance Evaluation of Machine Learning Methods
The numerical values of Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Pearson correlation are calculated according to the results of the training set, and the results are shown in Table 3. Furthermore, the values of each evaluation parameter are processed in the same direction, and the distribution and comparison of evaluation indexes of eight machine learning regression models are obtained (Figure 7). A higher value of the parameter reflects the better performance of the model. The performance parameters of integrated machine learning methods, such as RF and ANN, are higher than other models.
To minimize the influence of subjective factors, two objective weighting methods, the entropy weighting method and the critic method, are selected to calculate the weights of RMSE, MSE, MAE, and Pearson correlation. On this basis, the performance of eight machine methods is ranked by the TOPSIS algorithm to evaluate the performance of machine learning methods, as shown in Table 4 and Table 5. The weights of RMSE, MSE, MAE, and Pearson correlation calculated by entropy method are 0.2497, 0.2298, 0.2871, and 0.2334, respectively, while the weights of critic method are 0.1942, 0.902, 0.2012, and 0.4144, respectively.
According to the ranking results obtained by the entropy weight-TOPSIS method (Table 4) and critic weight algorithm-TOPSIS method (Table 5), eight FOS prediction models have almost the same results in the two evaluation models. The third to eighth positions in the final ranking of the models are as follows: GBDT, Bagging, Adaboost, KNN, SVM, and DT. ANN and RF are significantly superior to other models and the comprehensive scores of both evaluation models are greater than 0.84. If the comprehensive scores of the two evaluation models are added together, ANN has the best performance in terms of safety factor prediction.
5. Engineering Applications
5.1. Numerical Simulation of Slopes
Natural gas, as a kind of clean energy, has been widely used in civil and industrial fields due to its good economy and environmental protection. Piles AA001–AA004 of China–Russia East Line Natural Gas Pipeline is a controlled project of transit section, located in Heihe City, Heilongjiang Province. The geographical coordinates of Piles AA001–AA004 are between 127°21′24″ and 127°21′06″ east longitude and 50°17′30″ and 50°17′24″ north latitude. The monitoring area belongs to the landform of low mountains and hills. The ground elevation in the area is 95–290 m, while the relative elevation difference is 150–200 m. The cutting depth of the valley is 80–120 m. The monitoring area is characterized by cold temperate continental monsoon climate, with warm and rainy summer. The locations of the Pile AA001–AA004 and the monitoring slope are shown in Figure 8a. The slope angle is about 40°, the total length of the horizontal distance of the slope is 477 m, and the height difference is about 120 m. The slope body is mainly composed of a sand-gravel layer on the surface and strong weathered andesite. The surface of the monitoring area is exposed in a large area, and the surface layer is mainly composed of sandy gravel. The second layer is breccia and gravel layer, mainly consists of strong weathered andesite, with a particle size of 0.5–5 cm. The joints and fissures are relatively developed. The groundwater in the monitoring area is divided into two types, namely, loose accumulation pore water and bedrock fissure water. The loose accumulation pore water is recharged by atmospheric precipitation and river water in the region, which is greatly affected by seasons. The recharge is abundant in rainy seasons. Bedrock fissure water is only recharged by atmospheric precipitation. It is necessary to conduct a slope stability analysis since the stability of the slope covered by the natural gas pipeline directly affects the stability of oil and gas transportation.
According to the hydrogeological data in the field, the stability of the slope is evaluated and the FOS of the slope is calculated based on the numerical simulation method. A simple model of the landslide in the monitoring area is established in the leading software for slope stability calculation. The coordinates of the control points of each structural layer are input with the landslide foot as the origin, and the established model is shown in Figure 8b. The shear strength of rock and soil in the monitoring area of the first section of entry in Table 6 is used as the calculation parameter, and the calculation is carried out according to different moisture contents. The calculation method is to automatically search for the most dangerous sliding surface, and the step length is 1 m. After the dangerous sliding surface is determined, the step length is reduced to continue calculating the FOS. The safety factors are shown in Table 7.
In the situation of low water content (6%) and high water content (25%), the slope in the monitoring area tends to slide in shallow layers along the interface between soil and rock layers. The sliding scale is small and the sliding position is affected by Strong terrain control. In addition, the soil moisture content in the natural state is greater than its cohesion peak moisture content (10%), which means that the shear strength of the rock and soil mass will only decrease under the condition of rainfall.
5.2. Comparison of Numerical Simulation and Machine Learning Method Results
Table 8 gives the calculation of the FOS of the slope based on the ANN machine learning method and the input parameters of the slope. These data sets are used to verify the ANN method. The ANN method performs well in the calculation of the safety factor of the slope. Compared with the safety factor in Table 7, two of the results are very close to the numerical simulation results, namely, (1.05, 1.40) and (1.08, 1.35). With the increase in water content, the slope stability shows a trend of strengthening and then weakening, which is the same as the calculation result of numerical simulation.
This result difference is closely related to the historical data set. The existing training set does not cover the historical data of the slope close to the geometric parameters in this verification case, especially the slope height is quite different. The prediction results of the model can be further improved by more widely distributed data. Meanwhile, since the surface sandstone soil layer and the strongly weathered rock layer are considered, as a whole, to determine the FOS of the slope in the process of numerical simulation, the difference of the analysis perspective will also lead to the deviation of the calculation results.
6. Conclusions
This study applies machine learning and model evaluation methods to the analysis of slope stability. Based on 102 slope sample data, 6 parameters related to the arc-shaped slope instability are selected as sample features, and 8 regression machine learning methods are trained and verified to predict FOS. The prediction results of FOS by different machine methods show obvious differences.
The results of eight machine learning methods are evaluated by four accuracy evaluation indicators, including MSE, RMSE, MAE, and Pearson correlation, based on the entropy weight-TOPSIS and the CRITIC-TOPSIS models. The ANN and RF models show better performance in predicting FOS. Their comprehensive scores in the two evaluation methods are greater than 0.84, which is much higher than the comprehensive scores of other machine learning models. In addition, the ensemble methods show high accuracy.
The prediction results of ANN are used to compare the FOS of the Sino–Russian border natural gas pipeline calculated by the numerical simulation method. The obtained FOSs of slopes under different water content are close, while the error is related to the choice of data set and the difference in analysis methods, which proves the effectiveness and feasibility of the machine learning model.
Author Contributions
Conceptualization, G.B. and D.S.; methodology, G.B. and Z.T.; software, Y.Z.; validation, B.W., N.A. and Y.Y.; formal analysis, Y.Y. and M.Y.; investigation, Y.Z. and M.Y.; resources, G.B.; data curation, Y.Z. and N.A.; writing—original draft preparation, G.B. and D.S.; writing—review and editing, G.B. and D.S.; visualization, Z.T. and Y.Z.; supervision, Y.H.; funding acquisition, G.B. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by The 2021 Innovation Fund Project of Gansu Provincial Bureau of Geology and Mineral Resources, grant number 2021CX13.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Acknowledgments
The authors would like to thank the editor and reviewers for carefully dealing with this paper.
Conflicts of Interest
The authors declare no conflict of interest.
Nomenclature
| h | slope height, m |
| β | total slope angle, ° |
| γ | total slope angle, ° |
| c | cohesion, kPa |
| φ | internal friction angle, ° |
| ru | pore water ratio |
| FOS | safety factor |
| SVM | support vector machine |
| DT | decision tree |
| kNN | k-nearest neighbor algorithm |
| ADA | AdaBoost algorithm |
| RF | random forest |
| ANN | artificial neural network |
| Bagging | guided clustering algorithm |
| GBDT | gradient boosting decision tree |
| MSE | mean square error |
| RMSE | root mean square error |
| MAE | mean absolute error |
References
- Carrión-Mero, P.; Montalván-Burbano, N.; Morante-Carballo, F.; Quesada-Román, A.; Apolo-Masache, B. Worldwide Research Trends in Landslide Science. Int. J. Environ. Res. Public Health 2021, 18, 9445. [Google Scholar] [CrossRef]
- Chen, X.; Chen, Y.; Huang, Y.; Lv, X.; Xu, B. Comparative Analysis of Three Different Rigid Body Limit Equilibrium Methods. Yellow River 2016, 1, 116–119. [Google Scholar]
- Cao, Y. Comparative Study On Slope Stability Analysis of Foundation Pit Excavation. Jiangsu Constr. 2020, 2, 86–88. [Google Scholar]
- Yin, J.; Yang, J.; Ren, J.; Hu, D. Study of impact factors sensitivity of slope stability based on grey correlation analysis. J. Hefei Univ. Technol. 2015, 38, 75–78. [Google Scholar]
- Yibibula, M. Slope stability study based on the Swedish method and the Bishop method. Water Conserv. Sci. Technol. Econ. 2015, 21, 12–14. [Google Scholar]
- Hu, H.; Yao, L.; Dong, M. Comparison of slope stability evaluation with Swedish circle method and Bishop method. Subgrade Eng. 2007, 6, 110–112. [Google Scholar]
- Zhang, J.; Li, J. A Comparative Study Between Infinite Slope Model and Bishop’S Method for the Shallow Slope Stability Evaluation. Eur. J. Environ. Civ. Eng. 2021, 25, 1503–1520. [Google Scholar] [CrossRef]
- Dong, L.; Sun, D.; Li, X.; Zhou, Z. Interval Non-Probabilistic Reliability of a Surrounding Jointed Rockmass in Underground Engineering: A Case Study. IEEE Access 2017, 5, 18804–18817. [Google Scholar] [CrossRef]
- Dong, L.; Sun, D.; Li, X.; Ma, J.; Zhang, L.; Tong, X. Interval non-probabilistic reliability of surrounding jointed rockmass considering microseismic loads in mining tunnels. Tunn. Undergr. Sp. Tech. 2018, 81, 326–335. [Google Scholar] [CrossRef]
- Dong, L.; Sun, D.; Li, X. Theoretical and Case Studies of Interval Nonprobabilistic Reliability for Tailing Dam Stability. Geofluids 2017, 2017, 8745894. [Google Scholar] [CrossRef] [Green Version]
- Dong, L.; Shu, W.; Sun, D.; Li, X.; Zhang, L. Pre-Alarm System Based on Real-Time Monitoring and Numerical Simulation Using Internet of Things and Cloud Computing for Tailings Dam in Mines. IEEE Access 2017, 5, 21080–21089. [Google Scholar] [CrossRef]
- Dong, L.; Deng, S.; Wang, F. Some Developments and New Insights for Environmental Sustainability and Disaster Control of Tailings Dam. J. Clean. Prod. 2020, 269, 122270. [Google Scholar] [CrossRef]
- Zhang, J. Finite element simulation and stability analysis on slope excavation. Chin. J. Rock Mech. Eng. 2002, 21, 843–847. [Google Scholar]
- Wang, D.; Yuan, J. Slope research based on bishop method and strength reduction method. J. Yulin Univ. 2021, 31, 1–4. [Google Scholar]
- Chen, Q.; Hu, H.; Peng, J.; Hu, G. Visco-elastoplastic finite element simulation of progressive failure of high-steep rock slope. J. Eng. Geol. 2000, 8, 25–30. [Google Scholar]
- Li, S.; Li, H.; Shangguan, Z. Rock Slope Stability Under Earthquake Action with Finite Element Analysis. J. Liaoning Tech. Univ. 2015, 34, 1026–1030. [Google Scholar]
- Nian, T.; Huang, R.; Wan, S.; Chen, G. Three-Dimensional Strength-Reduction Finite Element Analysis of Slopes: Geometric Effects. Can. Geotech. J. 2012, 49, 574–588. [Google Scholar] [CrossRef]
- Zhang, Y.; Chen, G.; Zheng, L.; Li, Y.; Zhuang, X. Effects of Geometries on Three-Dimensional Slope Stability. Can. Geotech. J. 2013, 50, 233–249. [Google Scholar] [CrossRef]
- Dong, L.; Wesseloo, J.; Potvin, Y.; Li, X. Discrimination of Mine Seismic Events and Blasts Using the Fisher Classifier, Naive Bayesian Classifier and Logistic Regression. Rock Mech. Rock Eng. 2016, 49, 183–211. [Google Scholar] [CrossRef]
- Dong, L.; Tang, Z.; Li, X.; Chen, Y.; Xue, J. Discrimination of Mining Microseismic Events and Blasts Using Convolutional Neural Networks and Original Waveform. J. Cent. South Univ. 2020, 27, 3078–3089. [Google Scholar] [CrossRef]
- Peng, K.; Tang, Z.; Dong, L.; Sun, D. Machine Learning Based Identification of Microseismic Signals Using Characteristic Parameters. Sensors 2021, 21, 6967. [Google Scholar] [CrossRef]
- Dong, L.; Li, X.; Xie, G. Nonlinear Methodologies for Identifying Seismic Event and Nuclear Explosion Using Random Forest, Support Vector Machine, and Naive Bayes Classification. Abstr. Appl. Anal. 2014, 2014, 459137. [Google Scholar] [CrossRef] [Green Version]
- Dong, L.; Li, X.; Peng, K. Prediction of Rockburst Classification Using Random Forest. Trans. Nonferrous Met. Soc. China 2013, 23, 472–477. [Google Scholar] [CrossRef]
- Dong, L.; Shu, W.; Li, X.; Zhang, J. Quantitative Evaluation and Case Studies of Cleaner Mining with Multiple Indexes Considering Uncertainty Factors for Phosphorus Mines. J. Clean. Prod. 2018, 183, 319–334. [Google Scholar] [CrossRef]
- Lu, P.; Rosenbaum, M.; Chacon, J. Artificial Neural Networks and Grey Systems for the Prediction of Slope Stability. Nat. Hazards 2003, 30, 383–398. [Google Scholar] [CrossRef]
- Wang, H.; Xu, W.; Xu, R. Slope Stability Evaluation Using Back Propagation Neural Networks. Eng. Geol. 2005, 80, 302–315. [Google Scholar] [CrossRef]
- Li, X.; Dong, L.; Zhao, G.; Huang, M.; Liu, A.; Zeng, L.; Dong, L.; Chen, G. Stability Analysis and Comprehensive Treatment Methods of Landslides Under Complex Mining Environment—A Case Study of Dahu Landslide from Linbao Henan in China. Saf. Sci. 2012, 50, 695–704. [Google Scholar] [CrossRef]
- Das, S.K.; Biswal, R.K.; Sivakugan, N.; Das, B. Classification of Slopes and Prediction of Factor of Safety Using Differential Evolution Neural Networks. Environ. Earth Sci. 2011, 64, 201–210. [Google Scholar] [CrossRef]
- Liu, Z.; Shao, J.; Xu, W.; Chen, H.; Zhang, Y. An Extreme Learning Machine Approach for Slope Stability Evaluation and Prediction. Nat. Hazards 2014, 73, 787–804. [Google Scholar] [CrossRef]
- Zhou, J.; Li, E.; Yang, S.; Wang, M.; Shi, X.; Yao, S.; Mitri, H.S. Slope Stability Prediction for Circular Mode Failure Using Gradient Boosting Machine Approach Based on an Updated Database of Case Histories. Saf. Sci. 2019, 118, 505–518. [Google Scholar] [CrossRef]
- Wei, W.; Li, X.; Liu, J.; Zhou, Y.; Li, L.; Zhou, J. Performance Evaluation of Hybrid Woa-Svr and Hho-Svr Models with Various Kernels to Predict Factor of Safety for Circular Failure Slope. Appl. Sci. 2021, 11, 1922. [Google Scholar] [CrossRef]
- Qi, C.; Tang, X. Slope stability prediction using integrated metaheuristic and machine learning approaches: A comparative study. Comput. Ind. Eng. 2018, 118, 112–122. [Google Scholar] [CrossRef]
Figure 1.
Violin comparison diagram of the standardized data.
Figure 2.
Correlation analysis of the parameters.
Figure 3.
Results of the eight machine learning methods on training set.
Figure 4.
Results of the eight machine learning methods on test set.
Figure 5.
Results of the eight machine learning methods training set.
Figure 6.
Results of the eight machine learning methods test set.
Figure 7.
Evaluation index distribution radar diagram of the eight machine learning methods.
Figure 8.
(a) Locations of the Pile AA001–AA004 and the monitoring slope. (b) Stratigraphic model of the monitoring regional landslide.
Figure 8.
(a) Locations of the Pile AA001–AA004 and the monitoring slope. (b) Stratigraphic model of the monitoring regional landslide.
Table 1.
Database for training the machine learning methods.
| Case | γ (kN/m3) | c (kPa) | φ (°) | β (°) | h (m) | ru | FOS |
|---|---|---|---|---|---|---|---|
| 1 | 18.68 | 26.34 | 15.00 | 35.0 | 8.23 | 0.00 | 1.11 |
| 2 | 18.84 | 14.36 | 25.00 | 20.0 | 30.50 | 0.00 | 1.88 |
| 3 | 18.84 | 57.46 | 20.00 | 20.0 | 30.50 | 0.00 | 2.05 |
| 4 | 28.44 | 29.42 | 35.00 | 35.0 | 100.00 | 0.00 | 1.78 |
| 5 | 28.44 | 39.23 | 38.00 | 35.0 | 100.00 | 0.00 | 1.99 |
| 6 | 20.60 | 16.28 | 26.50 | 30.0 | 40.00 | 0.00 | 1.25 |
| 7 | 14.00 | 11.97 | 26.00 | 30.0 | 88.00 | 0.00 | 1.02 |
| 8 | 25.00 | 120.00 | 45.00 | 53.0 | 120.00 | 0.00 | 1.30 |
| 9 | 26.00 | 150.05 | 45.00 | 50.0 | 200.00 | 0.00 | 1.20 |
| 10 | 22.40 | 10.00 | 35.00 | 30.0 | 10.00 | 0.00 | 2.00 |
| 11 | 21.40 | 10.00 | 30.34 | 30.0 | 20.00 | 0.00 | 1.70 |
| 12 | 22.00 | 20.00 | 36.00 | 45.0 | 50.00 | 0.00 | 1.02 |
| 13 | 16.00 | 70.00 | 20.00 | 40.0 | 115.00 | 0.00 | 1.11 |
| 14 | 20.41 | 24.90 | 13.00 | 22.0 | 10.67 | 0.35 | 1.40 |
| 15 | 19.63 | 11.97 | 20.00 | 22.0 | 12.19 | 0.41 | 1.35 |
| 16 | 21.82 | 8.62 | 32.00 | 28.0 | 12.80 | 0.49 | 1.03 |
| 17 | 18.84 | 15.32 | 30.00 | 25.0 | 10.67 | 0.38 | 1.63 |
| 18 | 19.06 | 11.71 | 28.00 | 35.0 | 21.00 | 0.11 | 1.09 |
| 19 | 18.84 | 14.36 | 25.00 | 20.0 | 30.50 | 0.45 | 1.11 |
| 20 | 21.51 | 6.94 | 30.00 | 31.0 | 76.81 | 0.38 | 1.01 |
| 21 | 18.00 | 24.00 | 30.15 | 45.0 | 20.00 | 0.12 | 1.12 |
| 22 | 22.40 | 100.00 | 45.00 | 45.0 | 15.00 | 0.25 | 1.80 |
| 23 | 22.40 | 10.00 | 35.00 | 45.0 | 10.00 | 0.40 | 0.90 |
| 24 | 20.00 | 20.00 | 36.00 | 45.0 | 50.00 | 0.25 | 0.96 |
| 25 | 20.00 | 20.00 | 36.00 | 45.0 | 50.00 | 0.50 | 0.83 |
| 26 | 21.00 | 20.00 | 40.00 | 40.0 | 12.00 | 0.00 | 1.84 |
| 27 | 21.00 | 45.00 | 25.00 | 49.0 | 12.00 | 0.30 | 1.53 |
| 28 | 21.00 | 30.00 | 35.00 | 40.0 | 12.00 | 0.40 | 1.49 |
| 29 | 21.00 | 35.00 | 28.00 | 40.0 | 12.00 | 0.50 | 1.43 |
| 30 | 20.00 | 40.00 | 30.00 | 30.0 | 15.00 | 0.30 | 1.84 |
| 31 | 18.00 | 45.00 | 25.00 | 25.0 | 14.00 | 0.30 | 2.09 |
| 32 | 19.00 | 30.00 | 35.00 | 35.0 | 11.00 | 0.20 | 2.00 |
| 33 | 20.00 | 40.00 | 40.00 | 40.0 | 10.00 | 0.20 | 2.31 |
| 34 | 18.85 | 24.80 | 21.30 | 29.2 | 37.00 | 0.50 | 1.07 |
| 35 | 18.85 | 10.34 | 21.30 | 34.0 | 37.00 | 0.30 | 1.29 |
| 36 | 18.80 | 30.00 | 10.00 | 25.0 | 50.00 | 0.10 | 1.40 |
| 37 | 18.80 | 25.00 | 10.00 | 25.0 | 50.00 | 0.20 | 1.18 |
| 38 | 18.80 | 20.00 | 10.00 | 25.0 | 50.00 | 0.30 | 0.97 |
| 39 | 19.10 | 10.00 | 10.00 | 25.0 | 50.00 | 0.40 | 0.65 |
| 40 | 18.80 | 30.00 | 20.00 | 30.0 | 50.00 | 0.10 | 1.46 |
| 41 | 18.80 | 25.00 | 20.00 | 30.0 | 50.00 | 0.20 | 1.21 |
| 42 | 18.80 | 20.00 | 20.00 | 30.0 | 50.00 | 0.30 | 1.00 |
| 43 | 19.10 | 10.00 | 20.00 | 30.0 | 50.00 | 0.40 | 0.65 |
| 44 | 22.00 | 20.00 | 22.00 | 20.0 | 180.00 | 0.00 | 1.12 |
| 45 | 22.00 | 20.00 | 22.00 | 20.0 | 180.00 | 0.10 | 0.99 |
| 46 | 25.00 | 55.00 | 36.00 | 45.0 | 239.00 | 0.25 | 1.71 |
| 47 | 25.00 | 63.00 | 32.00 | 44.5 | 239.00 | 0.25 | 1.49 |
| 48 | 25.00 | 63.00 | 32.00 | 46.0 | 300.00 | 0.25 | 1.45 |
| 49 | 25.00 | 48.00 | 40.00 | 45.0 | 330.00 | 0.25 | 1.62 |
| 50 | 31.30 | 68.60 | 37.00 | 47.5 | 262.50 | 0.25 | 1.20 |
| 51 | 31.30 | 68.60 | 37.00 | 47.0 | 270.00 | 0.25 | 1.20 |
| 52 | 31.30 | 58.80 | 35.50 | 47.5 | 438.50 | 0.25 | 1.20 |
| 53 | 31.30 | 58.80 | 35.50 | 47.5 | 502.70 | 0.25 | 1.20 |
| 54 | 31.30 | 68.00 | 37.00 | 47.0 | 360.50 | 0.25 | 1.20 |
| 55 | 27.30 | 14.00 | 31.00 | 41.0 | 110.00 | 0.25 | 1.25 |
| 56 | 27.00 | 40.00 | 35.00 | 43.0 | 420.00 | 0.25 | 1.15 |
| 57 | 27.00 | 50.00 | 40.00 | 42.0 | 407.00 | 0.25 | 1.44 |
| 58 | 27.00 | 35.00 | 35.00 | 42.0 | 359.00 | 0.25 | 1.27 |
| 59 | 27.00 | 32.00 | 33.00 | 42.4 | 289.00 | 0.25 | 1.30 |
| 60 | 27.00 | 32.00 | 33.00 | 42.6 | 301.00 | 0.25 | 1.16 |
| 61 | 25.00 | 46.00 | 35.00 | 46.0 | 393.00 | 0.25 | 1.31 |
| 62 | 25.00 | 48.00 | 40.00 | 49.0 | 330.00 | 0.25 | 1.49 |
| 63 | 31.30 | 68.60 | 37.00 | 47.0 | 305.00 | 0.25 | 1.20 |
| 64 | 25.00 | 55.00 | 36.00 | 45.5 | 299.00 | 0.25 | 1.52 |
| 65 | 31.30 | 68.00 | 37.00 | 47.0 | 213.00 | 0.25 | 1.20 |
| 66 | 22.00 | 29.00 | 15.00 | 18.0 | 400.00 | 0.00 | 1.04 |
| 67 | 23.00 | 24.00 | 19.80 | 23.0 | 380.00 | 0.00 | 1.15 |
| 68 | 22.00 | 40.00 | 30.00 | 30.0 | 196.00 | 0.00 | 1.11 |
| 69 | 22.54 | 29.40 | 20.00 | 24.0 | 210.00 | 0.00 | 1.06 |
| 70 | 22.00 | 21.00 | 23.00 | 30.0 | 257.00 | 0.00 | 1.10 |
| 71 | 23.50 | 10.00 | 27.00 | 26.0 | 190.00 | 0.00 | 1.02 |
| 72 | 22.50 | 18.00 | 20.00 | 20.0 | 290.00 | 0.00 | 1.05 |
| 73 | 22.50 | 20.00 | 16.00 | 25.0 | 220.00 | 0.00 | 1.36 |
| 74 | 21.00 | 20.00 | 24.00 | 21.0 | 565.00 | 0.00 | 1.26 |
| 75 | 26.49 | 150.00 | 33.00 | 45.0 | 73.00 | 0.15 | 1.23 |
| 76 | 26.70 | 150.00 | 33.00 | 50.0 | 130.00 | 0.25 | 1.80 |
| 77 | 26.89 | 150.00 | 33.00 | 52.0 | 120.00 | 0.25 | 1.80 |
| 78 | 26.43 | 50.00 | 26.60 | 40.0 | 92.20 | 0.15 | 1.25 |
| 79 | 26.70 | 50.00 | 26.60 | 50.0 | 170.00 | 0.25 | 1.25 |
| 80 | 26.80 | 60.00 | 28.80 | 59.0 | 108.00 | 0.25 | 1.25 |
| 81 | 23.00 | 0.00 | 20.00 | 20.0 | 100.00 | 0.30 | 1.20 |
| 82 | 20.00 | 0.00 | 36.00 | 45.0 | 50.00 | 0.50 | 0.67 |
| 83 | 18.50 | 12.00 | 0.00 | 30.0 | 6.00 | 0.00 | 0.78 |
| 84 | 12.00 | 0.00 | 30.00 | 35.0 | 4.00 | 0.00 | 1.46 |
| 85 | 21.43 | 0.00 | 20.00 | 20.0 | 61.00 | 0.50 | 1.03 |
| 86 | 22.00 | 0.00 | 40.00 | 33.0 | 8.00 | 0.35 | 1.45 |
| 87 | 18.00 | 5.00 | 30.00 | 20.0 | 8.00 | 0.30 | 2.05 |
| 88 | 23.47 | 0.00 | 32.00 | 37.0 | 214.00 | 0.00 | 1.08 |
| 89 | 22.00 | 0.00 | 36.00 | 45.0 | 50.00 | 0.00 | 0.89 |
| 90 | 20.00 | 0.00 | 24.50 | 20.0 | 8.00 | 0.35 | 1.37 |
| 91 | 20.41 | 33.52 | 11.00 | 16.0 | 45.72 | 0.20 | 1.28 |
| 92 | 12.00 | 0.00 | 30.00 | 45.0 | 8.00 | 0.00 | 0.86 |
| 93 | 16.50 | 11.49 | 0.00 | 30.0 | 3.66 | 0.00 | 1.00 |
| 94 | 9.06 | 11.71 | 28.00 | 35.0 | 21.00 | 0.11 | 1.09 |
| 95 | 12.00 | 0.00 | 30.00 | 45.0 | 8.00 | 0.00 | 0.80 |
| 96 | 18.50 | 25.00 | 0.00 | 30.0 | 6.00 | 0.00 | 1.09 |
| 97 | 24.00 | 0.00 | 40.00 | 33.0 | 8.00 | 0.30 | 1.58 |
| 98 | 14.80 | 0.00 | 17.00 | 20.0 | 50.00 | 0.00 | 1.13 |
| 99 | 12.00 | 0.00 | 30.00 | 35.0 | 4.00 | 0.00 | 1.44 |
| 100 | 18.84 | 0.00 | 20.00 | 20.0 | 7.62 | 0.45 | 1.05 |
| 101 | 20.00 | 0.00 | 36.00 | 45.0 | 50.00 | 0.25 | 0.79 |
| 102 | 14.00 | 11.97 | 26.00 | 30.0 | 88.00 | 0.45 | 0.63 |
Table 2.
Optimal parameter of the eight machine learning algorithms.
| Model | Parameters |
|---|---|
| Random Forest | N_estimators = 10 Max_depth = 6 Min_samples_leaf = 1 Min_sanmples_split = 2 Criterion = ‘entropy’ |
| Decision Tree | Criterion = ‘gini’ Max_depth = 5 Ccp_alpha = 0.0 Min_samples_leaf = 1 Random_state = 111 |
| SVM | Kernel =‘poly’ Degree = 2 C = 100 Epsilon = 0.1 |
| GBDT | n_Estimators = 500 Max_Depth = 4 Min_Samples_Split = 2 Learning_Rate = 0.01 Loss = ‘Ls’ |
| KNN | n_Neighbors = 3 |
| AdaBoost | Max_depth = 2 Min_samples_split = 20 Min_samples_leaf = 5 Algorithm = ’SAMME’ N_estimators = 200 Learning_rate = 0.8 |
| Bagging | Max_samples = 0.5 Max_features = 0.5 |
| ANN | Activation = ’relu’ Neurons = (256, 128, 64, 32) Optimization_method = rmsprop Loss_function = ’mse’ Metrics = ’mae’ |
Table 3.
Evaluation results of the machine learning methods.
| Model | MSE | RMSE | MAE | Pearson Correlation |
|---|---|---|---|---|
| RF | 0.0725 | 0.2587 | 0.1925 | 0.3272 |
| DT | 0.1205 | 0.3413 | 0.2397 | 0.2741 |
| SVM | 0.1088 | 0.3174 | 0.2600 | 0.2871 |
| GBDT | 0.0720 | 0.2648 | 0.1950 | 0.2892 |
| KNN | 0.1086 | 0.3129 | 0.2442 | 0.2882 |
| Adaboost | 0.1004 | 0.3129 | 0.2463 | 0.2953 |
| Bagging | 0.0878 | 0.2950 | 0.2299 | 0.2956 |
| ANN | 0.0661 | 0.2514 | 0.1882 | 0.2987 |
Table 4.
Ranking results based on entropy weight-TOPSIS method.
| Model | Positive Ideal Solution | Negative Ideal Solution | Comprehensive Score | Ranking |
|---|---|---|---|---|
| RF | 0.0214 | 0.1149 | 0.8432 | 2 |
| DT | 0.1289 | 0.0135 | 0.0948 | 8 |
| SVM | 0.1185 | 0.0189 | 0.1378 | 7 |
| GBDT | 0.0315 | 0.1090 | 0.7759 | 3 |
| KNN | 0.1125 | 0.0231 | 0.1703 | 6 |
| Adaboost | 0.1034 | 0.0322 | 0.2376 | 5 |
| Bagging | 0.0774 | 0.0576 | 0.4267 | 4 |
| ANN | 0.0165 | 0.1316 | 0.8886 | 1 |
Table 5.
Ranking results based on critic-TOPSIS method.
| Model | Positive Ideal Solution | Negative Ideal Solution | Comprehensive Score | Ranking |
|---|---|---|---|---|
| RF | 0.0188 | 0.1052 | 0.8482 | 1 |
| DT | 0.1175 | 0.0113 | 0.0877 | 8 |
| SVM | 0.1061 | 0.0185 | 0.1484 | 7 |
| GBDT | 0.0354 | 0.0958 | 0.7301 | 3 |
| KNN | 0.1011 | 0.0219 | 0.1782 | 6 |
| Adaboost | 0.0923 | 0.0310 | 0.2513 | 5 |
| Bagging | 0.0701 | 0.0522 | 0.4270 | 4 |
| ANN | 0.0220 | 0.1163 | 0.8410 | 2 |
Table 6.
Shear strength of rock mass varying with water content.
| Sand and Soil Surface | Strongly Weathered Andesite | |||||||
|---|---|---|---|---|---|---|---|---|
| Water content (%) | 6 | 10 | 20 | 25 | 6 | 10 | 20 | 25 |
| c (kPa) | 8 | 85 | 17 | 9 | 215 | 195 | 151 | 130 |
| φ (°) | 30 | 25 | 25 | 17.9 | 15.6 | 14.6 | 12.7 | 11.9 |
| γ (kN/m3) | 13.2 | 13.75 | 15 | 15.63 | 15.75 | 16.35 | 17.82 | 18.56 |
Table 7.
Prediction results of FOS.
| Water Content (%) | Radius of Dangerous Sliding Surface (m) | Sliding Force (kN) | Anti-Sliding Force (kN) | FOS |
|---|---|---|---|---|
| 6 | 102.9 | 1236 | 1802 | 1.458 |
| 10 | 184.53 | 76,345.938 | 145,378 | 1.904 |
| 25 | 102.91 | 1458.175 | 1677.549 | 1.150 |
Table 8.
Safety factor value based on ANN method.
| Water Content (%) | Slope Layer | γ (kN/m3) | c (kPa) | φ (°) | ß (°) | h (m) | ru | FOS |
|---|---|---|---|---|---|---|---|---|
| 6 | Sand-gravel layer | 13.20 | 8 | 8 | 40 | 120 | 0 | 1.05 |
| Strong Weathered Andesite | 15.75 | 215 | 215 | 40 | 120 | 0 | 1.40 | |
| 10 | Sand-gravel layer | 13.75 | 85 | 85 | 40 | 120 | 0 | 1.38 |
| Strong Weathered Andesite | 16.35 | 195 | 195 | 40 | 120 | 0 | 1.40 | |
| 25 | Sand-gravel layer | 15.63 | 9 | 9 | 40 | 120 | 0 | 1.08 |
| Strong Weathered Andesite | 18.56 | 130 | 130 | 40 | 120 | 0 | 1.35 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Bai, G.; Hou, Y.; Wan, B.; An, N.; Yan, Y.; Tang, Z.; Yan, M.; Zhang, Y.; Sun, D. Performance Evaluation and Engineering Verification of Machine Learning Based Prediction Models for Slope Stability. Appl. Sci. 2022, 12, 7890. https://doi.org/10.3390/app12157890
AMA Style
Bai G, Hou Y, Wan B, An N, Yan Y, Tang Z, Yan M, Zhang Y, Sun D. Performance Evaluation and Engineering Verification of Machine Learning Based Prediction Models for Slope Stability. Applied Sciences. 2022; 12(15):7890. https://doi.org/10.3390/app12157890
Chicago/Turabian StyleBai, Gexue, Yunlong Hou, Baofeng Wan, Ning An, Yihao Yan, Zheng Tang, Mingchun Yan, Yihan Zhang, and Daoyuan Sun. 2022. "Performance Evaluation and Engineering Verification of Machine Learning Based Prediction Models for Slope Stability" Applied Sciences 12, no. 15: 7890. https://doi.org/10.3390/app12157890
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
