Assessment of Machine Learning Models for the Prediction of Rate-Dependent Compressive Strength of Rocks

Abstract

The prediction of rate-dependent compressive strength of rocks in dynamic compression experiments is still a notable challenge. Four machine learning models were introduced and employed on a dataset of 164 experiments to achieve an accurate prediction of the rate-dependent compressive strength of rocks. Then, the relative importance of the seven input features was analyzed. The results showed that compared with the extreme learning machine (ELM), random forest (RF), and the original support vector regression (SVR) models, the correlation coefficient R² of prediction results with the hybrid model that combines the particle swarm optimization (PSO) algorithm and SVR was highest in both the training set and the test set, both exceeding 0.98. The PSO-SVR model obtained a higher prediction accuracy and a smaller prediction error than the other three models in terms of evaluation metrics, which showed the possibility of the model as a rate-dependent compressive strength prediction tool. Additionally, besides the static compressive strength, the stress rate is the most important influence factor on the rate-dependent compressive strength of the rock among the listed input parameters. Moreover, the strain rate has a positive effect on the rock strength.

1. Introduction

Strain rate is one of the most important factors affecting the dynamic properties of rocks [1]. Especially for engineering projects involving blasting and excavation, the stress waves generated by blasting are different from the static loads acting on rocks and are complex dynamic processes. In the blasting analysis of rock tunnels, the dynamic properties of rocks show obvious strain rate dependence, and the effect of strain rate should be further considered [2,3,4]. Many studies have shown that the mechanical properties of rock materials change significantly as the strain rate increases [5,6]. Therefore, it is of great theoretical and practical importance to study the compressive strength of rocks under different strain rate conditions. Moreover, understanding the rate dependence of rock strength is of great importance to rock engineering design and construction [7,8].

As a key indicator of the mechanical properties of rocks, the current methods for determining the compressive strength of rocks rely on laboratory tests such as static compression tests and the Split Hopkinson Pressure Bar (SHPB) test [9,10]. These test methods investigate the rate dependence of the compressive strength of rocks at low and high strain rates, respectively. However, these methods are cumbersome and time-consuming to operate. Moreover, it is difficult to use these test methods to directly study the compressive strength of rocks at moderate strain rates. The study of the mechanical properties of rocks at moderate strain rates can be of great help in understanding the mechanism of excavation-induced geohazards (e.g., rock bursts) [11,12,13]. Some scholars have attempted to modify the test setup to control the intermediate strain rate [14], but it remains a challenging task. This is because there are many influencing factors, and these factors are complexly coupled with each other. Although numerical simulations have been applied in this area [15,16,17], the accurate prediction of rock strength depends on a reasonable intrinsic model and reliable model calibration. Therefore, there is an urgent need for a simple, fast, and reliable intelligent method to predict the compressive strength of rocks at different strain rates and to facilitate understanding of the rate dependence of rock strength.

With the development of artificial intelligence, some intelligent methods such as machine learning have been used to solve some complex engineering problems. In recent years, machine learning techniques such as artificial neural network (ANN), support vector machine (SVM), random forest (RF), extreme learning machine (ELM), and other models have been employed to predict concrete compressive strength and have achieved good prediction behavior [18,19,20,21,22,23,24,25]. Similarly, some intelligent models have been introduced to predict the mechanical properties and stability of rocks, with significant progress being made by numerous scholars [26,27,28,29,30,31,32,33,34]. For example, Ebrahim, et al. [35] developed a model tree approach to predict the uniaxial compressive strength and elastic modulus of carbonate rocks, which provided better prediction results. Li, et al. [36] utilized the Least squares support vector machine model to predict the compressive strength and shear strength of rocks, and the correlation coefficient R² of the predicted results exceeded 0.99. Yılmaz, et al. [37] employed an ANN to predict the compressive strength and elastic modulus of rocks. Compared with the conventional statistical model, the ANN network obtained a higher prediction accuracy. Ehsan, et al. [38] proposed a particle swarm optimization (PSO) algorithm-based ANN model with four input parameters including point load index, Schmidt hammer rebound number, P-wave velocity, and dry density, to predict the uniaxial compressive strength of rocks. Compared with the conventional ANN, the hybrid optimized model reached higher prediction accuracy with R² = 0.97. Gupta et al. [39] employed five machine learning models on 170 samples to predict rock strength, and the proposed density-weighted least squares twin support vector regression (SVR) model showed the best predictive performance compared with the other four models in terms of evaluation indicators. Hany et al. [40] developed two models, random forest and functional network, to predict the unconstrained compressive strength of rocks using six parameters: drilling torque, weight on bit, mud pumping rate, stand-pipe pressure, drill string rotating speed, and the rate of penetration. The model results showed that the developed RF and functional network models can provide accurate uniaxial compression strength estimations from drilling data in real-time. The application of the model saves time and costs and provides data support and guidance to improve well stability.

Although many successful applications have been achieved using methods such as empirical formulas [41] and intelligent models to estimate the compressive strength of rocks [42,43], it is undeniable that the relevant research has mainly focused on static compressive strength, less research has been performed on the rate-independent compressive strength of rocks, and more research is needed to improve the understanding of this problem. In particular, further analysis is required to understand the compressive strength of rocks under rate independence to improve the ability to predict the rate dependence of rock strength. Fortunately, these successful applications provide a research base and important guidance for the further extension of machine learning model applications to rock dynamics. To this end, this paper attempts to propose a hybrid model which combines a PSO algorithm and SVR to predict the rate-independent compressive strength of rocks. In addition, a comparative analysis with three other models (ELM, random forest, and SVR) is performed, providing new insights into the rate-dependence of rock strength.

2. Method and Models

2.1. Extreme Learning Machine

ELM is an algorithm based on a single hidden layer feedforward neural network, in which the input weights and biases are randomly assigned, and the output weights are calculated using the Moore–Penrose generalized inverse within the framework of the least-squares criterion [44,45,46]. Therefore, ELM has the advantage of fast convergence and is less likely to fall into local extremes than traditional neural networks based on gradient descent learning theory [47,48]. The network architecture of ELM is shown in Figure 1. Given a dataset containing N arbitrary samples (x_i, t_i), the number of nodes in the input layer is n and the number of nodes in the output layer is m, where $x_i=\left[x_{i1},x_{i2},\cdots ,x_{in}\right]\in R^n$, $t_i=\left[t_{i1},t_{i2},\cdots ,t_{im}\right]\in R^m$. For a neural network with an activation function g(x) and a single hidden layer with K hidden nodes, the expression is shown below.

$${\displaystyle \sum }_{i=1}^K{\beta }_{i}g\left(⟨w_i,x_i⟩+b_i\right)=o_j j=1,2,\cdots ,N$$

(1)

where $w_i={[w_{i1},w_{i2},\cdots ,w_{in}]}^T$ is the weight vector between the ith hidden node and the input node; ${\beta }_i={[{\beta }_{i1},{\beta }_{i2},\cdots ,{\beta }_{in}]}^T$ is the weight vector between the ith hidden node and the output node; b_i is the bias of the ith hidden node; $⟨w_i,x_i⟩$ is the inner product of w_i and x; o_j is the output value.

It is known that the learning goal of a single hidden layer neural network is to minimize the error in the output, i.e., there exists β_i, w_i, and b_i satisfying the following conditions.

$${\displaystyle \sum }_{i=1}^K{\beta }_{i}g\left(⟨w_i,x_i⟩+b_i\right)=t_j j=1,2,\cdots ,N$$

(2)

Equation (2) is expressed in matrix form as follows.

$$H\beta =T$$

(3)

2.2. Random Forest

Random forest is a variant of the bagging integration algorithm to improve the model. The random forest uses decision trees as the base learner and builds a random forest model by integrating several decision trees, while the random forest introduces a random selection of feature attributes in the training process of the decision trees [49,50]. Based on this mechanism, random forest inherits the advantages of sample perturbation from the bagging integration algorithm and improves on it by introducing the perturbation strategy of a random selection of attributes. Therefore, for the same data set, the double randomness of the random forest provides a better generalization ability and overfitting resistance. As shown in Figure 2, the algorithm principle of random forest can be expressed as follows.

$$Y=\frac{1}{N}{\displaystyle \sum }_{i=1}^N{F}_i\left(X\right)$$

(4)

where X is the input feature vector, Y is the prediction result, and N is the number of regression tree models built.

2.3. Support Vector Regression

Support vector machines can effectively solve classification and complex nonlinear regression problems [52]. When they are applied to regression problems, the basic idea is to find an optimal classification surface that minimizes the error of all training samples from that classification surface [53,54,55]. Assuming that there is a set of training samples $\left\{\left(x_i,y_i\right),i=1,2,\cdots ,l\right\}\in \left(R^n\times R\right)$, a linear regression function is established in the high-dimensional feature space as follows.

$$f\left(x\right)=w\varphi \left(x\right)+b$$

(5)

where $\varphi \left(x\right)$ is a nonlinear mapping function. By finding w as well as b according to the structural risk minimization principle, the problem of solving the regression function is transformed into the optimization problem of the function [56,57].

$$\left\{\begin{array}{l}\min \frac{1}{2}‖w^2‖+C{\displaystyle \sum _{i=1}^l({\xi }_i+{\xi }_i^{*})}\\ s.t.\left\{\begin{array}{l}{y}_i-w\varphi (x)-b\le \epsilon +{\xi }_i\\ -y_i+w\varphi (x)+b\le \epsilon +{\xi }_i^{*}\\ {\xi }_i,{\xi }_i^{*}\ge 0\end{array}\right.\end{array}\right.$$

(6)

where C is the penalty factor. A larger C indicates a larger penalty for samples with a training error greater than ε; ε specifies the error requirement of the regression function, and a smaller ε indicates a smaller error of the regression function. ${\xi }_i$ and ${\xi }_i^{*}$ are slack variables.

This optimization problem can be solved by introducing the Lagrange function and transforming it into a pairwise form. The regression function can eventually be expressed as:

$$f\left(x\right)={\displaystyle \sum }_{i=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)K\left(x_i,x\right)+b$$

(7)

where $K\left(x_i,x\right)=\varphi \left(x_i\right)\varphi \left(x\right)$ is the kernel function. In this paper, the radial basis function (RBF) with a wide convergence domain is selected as the kernel function.

2.4. Support Vector Regression with Particle Swarm Optimization

For the SVR model with RBF function, the combined values of C and g have a significant effect on the predictive ability of the model. Some intelligent algorithms such as grid search, genetic algorithm, and PSO algorithm, are used to optimize the model. Considering the advantages of the PSO algorithm with fewer parameters and higher efficiency, a PSO-based SVR model is proposed to improve the model performance of the original SVR. The PSO algorithm can be represented as follows. In addition, the training and testing process of the hybrid model is shown in Figure 3.

$$\left\{\begin{array}{l}{v}_i^{k+1}=\omega ·v_i^k+c_1{r}_1(pbes{t}_i^k-x_i^k)+c_2{r}_2(gbes{t}_i^k-x_i^k)\\ x_i^{k+1}=x_i^k+v_i^{k+1}\end{array}\right.$$

(8)

where ω is the initial weight, k is the number of iterations, $v_i^k$ and $x_i^k$ are the velocity and position vectors of the particle, respectively, c₁ and c₂ are the learning factors, r₁ and r₂ are arbitrary values between [0, 1], $pbes{t}_i^k$ is the best position passed by the i-th particle, and $gbes{t}_i^k$ is the global best position.

3. Dataset Description

A total of 164 datasets were collected from the literature [5]. Each data set consists of seven input parameters which include length, diameter, grain size, bulk density, P-wave velocity, strain rate, and static compressive strength (SCS). The output is the rate-dependent compressive strength (CS) of the rocks. The distribution between each input parameter and output parameter is shown in Figure 4. To further understand the characteristics of the input parameters, the distribution characteristics of these parameters are listed in

Table 1. Distribution characteristics of the parameters.

Parameter	Length	Diameter	Grain Size	Bulk Density	P-Wave Velocity	Strain Rate	SCS	CS
Unit	mm	mm	mm	kg/m3	m/s	s−1	MPa	MPa
Max	70	70	3.5	2850	6651	223	212	352.71
Min	10	2.5	0.03	2278	2437	0.000005	28.6	30.03
Mean	40.43	41.95	0.75	2479.68	3542.99	59.48	87.55	128.14
Median	49.77	49.50	0.23	2384.00	3031.00	56.00	71.91	101.42
Standard deviation	15.81	18.15	1.07	186.21	1231.89	46.85	56.39	80.23
Coefficient of variation	0.39	0.43	1.43	0.08	0.35	0.79	0.64	0.63
Kurtosis	−0.74	−0.55	1.76	−1.04	0.12	1.16	−0.81	−0.32
Skewness	−0.76	−0.62	1.75	0.59	1.33	0.90	0.75	0.91
Pearson correlation coefficient	−0.22	0.13	0.54	0.71	0.78	0.16	0.89	1

. For model training and testing, 131 sets of data were randomly selected for the training of the model, and the remaining data set was used as the test set. For comparison purposes, the training set and test set will be fixed after the training set has been randomly selected by the first model. In this way, the training and test sets of the four models are identical. The model training phase used 10-fold cross-validation.

In addition, three evaluation metrics are introduced for quantification when assessing and comparing the predictive performance of the models. The definitions of these evaluation metrics are listed in

Table 2. Definitions of evaluation indexes.

Evaluation Metrics	Definition
Correlation coefficient	$R^2={\frac{(n{\displaystyle \sum _{i=1}^n(O_e{O}_p)}-{\displaystyle \sum _{i=1}^n{O}_e{\displaystyle \sum _{i=1}^n{O}_p)2}}}{[n({\displaystyle \sum _{i=1}^n{O}_e^2})-{({\displaystyle \sum _{i=1}^n{O}_e})}^2][n({\displaystyle \sum _{i=1}^n{O}_p^2})-{({\displaystyle \sum _{i=1}^n{O}_p})}^2]}}^{}$
Mean absolute error	$MAE=\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}\left|O_e-O_p\right|$
Mean absolute percentage error	$MAPE=\frac{1}{n}{\displaystyle {\displaystyle \sum }_{i=1}^n}\left|\frac{O_e-O_p}{O_e}\right|\times 100\%$

[58,59,60,61].

Where O_e and O_p are the true and predicted result of the rate-dependent compressive strength, respectively.

4. Model Performance Comparison

After training and testing, the prediction results of the four models are shown in Figure 5. Moreover, a linear fitting was performed between the predicted results and the actual values. According to the correlation coefficient R² provided in Figure 5, it can be seen that among the four models, the PSO-SVR model achieves the best prediction performance, followed by the random forest, SVR, and ELM model. Specifically, the PSO-SVR model presents the best prediction results both in the training and testing phases. For both the training and test sets, the correlation coefficient R² of the models exceeded 0.98.

The box line plots of the relative prediction errors for the testing samples are shown in Figure 6. Compared with the ELM model, the mean prediction relative errors of the other three models were within 10%. For the PSO-SVR model, the mean value of the prediction error reached a minimum of 7.944%. To facilitate further comparison of the prediction performance of different models,

Table 3. Evaluation indicators of four models.

Model	MAE	MAPE/%	R2
ELM training	12.531	10.620	0.947
ELM test	15.573	12.664	0.946
RF training	9.429	8.411	0.972
RF test	10.312	9.007	0.979
SVR training	10.218	9.871	0.969
SVR test	11.04145	9.278841	0.972
PSO-SVR training	4.9511	4.718	0.992
PSO-SVR test	10.052	7.944	0.980

lists the evaluation metrics for the training and testing phases of each model. For the PSO-SVR model, the correlation coefficient R² is the highest and is close to 1. The other two error metrics are smaller than the other three models, which confirm the high accuracy, reliability, and generalization ability of the PSO-SVR model in predicting the rate-dependence compressive strength of rocks.

5. Relative Importance of Input Parameters

Analyzing the influence degree and relative importance of each influencing parameter on the output results is important for the prediction of the results. In particular, it is important to understand the positive or negative effect of each influencing parameter on the output. Figure 7 shows the relative importance of each influence parameter on the output results. It can be seen that among the seven influence parameters listed, the static compressive strength is the most important parameter, with a relative importance of more than 75%. It is followed by the strain rate, whose relative importance is about 15%. The remaining five parameters have a smaller degree of influence on the output results. Further, the effect of each influencing parameter on the output results is analyzed in detail and presented in Figure 8. Each scatter in Figure 8 represents a rock sample in the data set. The red color indicates that the parameter is positive for the output, while the blue color indicates that it is negative for the output. It can be observed that the three parameters, static compressive strength, strain rate, and bulk density, have a significant positive effect on the rate-independent compressive strength of rocks. Their increase leads to an increase in compressive strength.

6. Conclusions

In this work, four intelligent machine learning models were introduced to predict the rate-independent compressive strength of rocks. The main findings are summarized below.

(1)

All four machine learning models presented in this paper can effectively achieve a fast and rough estimation of the rate-independent compressive strength of rocks for a given combination of input parameters. Compared with ELM, the average relative prediction errors of the random forest, SVR, and PSO-SVR models were all within 10%, while the PSO-SVR model reached a minimum average relative error of 7.944% for the test set.

(2)

The PSO-SVR model could capture the complex nonlinear mapping between multiple inputs and outputs more accurately than the other two models in terms of the evaluation metrics, and its prediction performance is superior to the other three methods.

(3)

Among the seven input parameters mentioned, the static compressive strength and the strain rate are the two most important variables for the rate-independent compressive strength of rocks. The three parameters, static compressive strength, strain rate, and bulk density, have significant positive effects on the rate-independent compressive strength of rocks. Their increase leads to an increase in compressive strength.