On the Use of Neuro-Swarm System to Forecast the Pile Settlement

Abstract

In civil engineering applications, piles (deep foundations) are pushed into the ground in order to perform as steady support of structures. As these type of foundations are able to carry a huge amount of load, they should be carefully designed in terms of their settlement. Therefore, the control and estimation of settlement is a significant issue in pilling design and construction. The objective of the present study is to introduce a modeling process of a hybrid intelligence system namely neural network optimized by particle swarm optimization (neuro-swarm) for estimation of pile settlement. To do that, properties results of several piles socketed into rock mass together with their settlements were considered as established databased to propose neuro-swarm model. Then, several sensitivity analyses were carried out to determine the most influential particle swarm optimization parameters for pile settlement prediction. Eventually, five neuro-swarm models were constructed to understand the behavior of this hybrid model on them in pile settlement prediction. As a result, according to results of five performance indices, dataset number 4 showed the highest prediction capacity among all five datasets. The coefficient of determination (R²) and system error values of (0.851 and 0.079) and (0.892 and 0.099) were obtained respectively for train and test stages of the best neuro-swarm model which reveal the capability level of this hybrid model in predicting pile settlement. The modeling process introduced in this study can be useful for the researchers who are interested to work on the same hybrid technique.

1. Introduction

Structural loads are carried by rock-socketed piles via their end bearing, or their shaft resistance, or a mixture of both. As stated by Carrubba [1], the application of drilled piles socketed into rock is among the best alternatives when bedrock is overlain by layers of loose soil at shallow depths. In such circumstances, shaft resistance in rock can ensure considerable bearing capacity, even with small pile dislocations. Ng et al. [2] mentioned that there is a pressing need to provide rock-socketed piles for enhancing the efficacy of pile designing and greater design loads. The design of such structures is performed through experiential or analytical approaches [3]. The latter has received substantial interest, and finite-element codes are currently accessible. The former is normally established upon the outcomes of full-scale analysis of piles. As stated by Carrubba [1], the two procedures have equal suitability in the design of rock-socketed piles. Even so, nearly all the accessible methods do not predict accurately due to the complicated action of piles. Our research, therefore, aims to present a novel intelligent model for forecasting the settlement of rock-socketed piles featuring the uppermost importance in pile designing. Poulos [4] presented evidence in the field of predicting pile capacity and the findings were merely appropriate for input information. Thus, the succeeding section reviews the literature in order to identify the most appreciate model inputs for pile settlement estimation. Based on a research carried out by Randolph and Wroth [5], settlement of the pile is dependent upon pile geometry parameters (e.g., diameter, area and length), shear modulus of soil and piling load. Regarding rock-socketed piles, numerous investigators [6,7], have shown that the uniaxial compressive strength (UCS) of the rocks has a direct relationship with the pile bearing capacity and its settlement. A popular intelligence system namely the artificial neural network (ANN) was designed and introduced in study by Pooya Nejad et al. [8] to predict piling settlement using the standard penetration test data. They utilized nearly 1000 data, collected from previous studies, to develop the ANN models and concluded that the ANN is a powerful technique for solving problem related to pile settlement. In another intelligence study of foundation settlement, Soleimanbeigi and Hataf [9] introduced an ANN model to estimate the settlement of spread foundations using soil properties, the foundation geometry parameters and reinforcement specifications. In a similar study, Shahin et al. [10] successfully demonstrated the applicability of ANN for estimation of the spread foundations settlement. According to their results, the most effective parameters on the spread foundations settlement were standard penetration test (SPT) blow count (N), foundation embedded ratio and foundation geometry (like width of foundation). Chan et al. [11] employed an ANN model containing three layers in order to examine the driven piles bearing capacity with the help of the input parameters i.e., the elastic compression of soil and pile, pile setup and generating energy used in a pile. Abu Kiefa [12] attempted to propose a regression ANN capable of estimating a driven pile capacity within non-cohesive soil. He set four parameters as model inputs to his model, namely the soil friction angle, effective overburden pressure, duration of pile and the cross-sectional region of the pile. Ardalan et al. [13] made use of the group method of data handling (GMDH) model optimized by genetic algorithm (GA). A gene expression programming (GEP) equation has been proposed in another relevant research by Alkroosh and Nikraz [14] to estimate the axial capacity of driven piles. Their model utilized a number of independent variables such as the length and size of piles, weighted sleeve friction along the pile shaft, pile materials and pile elastic modulus. A GEP equation was developed by Armaghani et al. [3] to approximate settlement of the rock-socketed piles. Armaghani et al. [3] successfully showed that their GEP model is able to perform better than statistical model. Some other techniques e.g., particle swarm optimization (PSO)-ANN, GA-ANN, imperialism competitive algorithm (ICA)-ANN, adoptive neuro-fuzzy inference system-GMDH-PSO and genetic programming were introduced for pile bearing capacity prediction, in the studies carried out by Armaghani et al. [7], Momeni et al. [15], Moayedi and Armaghani [16], Harandizadeh et al. [17] and Chen et al. [18], respectively. As reported in the literature, piling loads, UCS of the rock, piling geometry parameters, pile length in different layers (i.e., soil and rock) and SPT-N value are model inputs of reasonable importance to predict the settlement of rock-socketed piles.

Furthermore, several studies [18,19,20,21,22,23,24,25,26,27,28,29,30,31], have reported the successful application of artificial intelligent (AI) and machine learning (ML) techniques such as ANNs to approximate and consequently solve piling problems as well as civil engineering problems. Despite using ANNs in literatures in order to resolving various problems in solving science and engineering problems, some investigations (e.g., [32,33,34,35]) confirmed that ANN has several inherent limitations such as lagging in learning rate and getting trapped in local minima. Therefore, using optimization algorithms (OAs) such as ICA and PSO show significant gain in order to dominate these restrictions and allow promotion of models predicted by adjusting bias and weight of ANNs. Recently, considerable capacity of these algorithms drew more attentions and researchers tried to combine them in order to optimize ANN models [26,36,37,38]. In piling technology, several studies have been conducted to solve the problems by developing PSO-ANN models. For example, Armaghani et al. [7] conducted a study of prediction of pile bearing capacity using a combination of PSO and ANN models. They predicted pile bearing capacity through the use of the most important parameters on the pile capacity and showed that the prediction performance of the PSO-ANN model is higher than the performance prediction of ANN model. A research on the load-deformation behavior of axially loaded piles was conducted by Ismail et al. [39]. They developed a PSO-ANN technique to predict the behavior of single piles embedded in soil and subjected to an axial load. Pile load, pile diameter, pile modulus, soil stiffness and pile length were used as inputs whereas pile load-deformation was considered as system output. In another study, the PSO was used to optimize ANN weights and biases in predicting lateral deflection of piles by Khari et al. [40]. They successfully indicated that their developed hybrid model is able to provide a high level of accuracy. Apart from the examples of PSO-ANN implementation studies, PSO has been used for optimization purposes in many geotechnical applications (e.g., [41,42,43]).

In this study, the development of hybrid ANN-based model namely PSO-ANN (or neuro-swarm) with detailed modeling process is presented herein for estimating the settlement of rock-socketed piles. For comparison purposes, a pre-developed ANN model is also developed to predict the settlement of piles. Then, the performance prediction of these techniques is evaluated and the best model among them is selected. The rest of this study is organized in the following way. In Section 2, the background of ANN, PSO and PSO-ANN models is described. Then, after giving some descriptions about database and study area, development process and the used models will be given in detail. Eventually, their prediction capacities are evaluated to introduce the best predictive model for estimation of the pile settlement.

2. Methods and Material

2.1. Artificial Neural Network (ANN)

ANN can be formed and expanded for data processing attributable to the structure of a human brain [44]. An ANN structure comprises three major parts, namely input, output and hidden layers. In each network layer, data is transmitted among layers by binding elements, such as neurons. Then, the weights created by the system, take control of this transmission for strengthening or weakening processes. An activation function (sometimes sigmoid type) needs to be used for the calculation of the output of each neuron in the layer [45,46]. The No. of inputs determines the No. of neurons available in every layer. Typically, to obtain the number of neurons, one can implement a complicated approach or a trial-and-error process [47,48]. The back propagation (BP) training algorithm has been applied more frequently—amongst all network training algorithms—in the field of engineering [49,50,51,52]. Furthermore, network creation and determination of corresponding weight constitute ANN modeling. The BP training algorithm is implemented to regulate network weights according to the minimization of the error value [21,53,54]. A comparison is made between the desired output values with the values reached each stage. The process calumniates once the desired values are obtained and the system error is diminished [55,56,57].

2.2. Particle Swarm Optimization (PSO)

Kennedy and Eberhart [58] first developed the PSO and afterwards it has been updated several times by other researchers (e.g., [59]). Compared to GA, PSO takes advantage of high learning speed and requires less memory. In PSO, particles search the best global (G_best) and the best personal (P_best) positions to reach the superior position [60]. On the other hand, a particle moves to find the best positions (P_best and G_best) in each iteration of the system. The following relations are used to obtain the velocity and position of particles.

$${\mathrm{V}}_{\mathrm{new}}=\mathrm{w}\ast \mathrm{V}+{\mathrm{C}}_1.{\mathrm{r}}_1\left({\mathrm{P}}_{\mathrm{best}}-\mathrm{X}\right)+{\mathrm{C}}_2.{\mathrm{r}}_2\left({\mathrm{G}}_{\mathrm{best}}-\mathrm{X}\right)$$

(1)

$${\mathrm{X}}_{\mathrm{new}}=\mathrm{X}+{\mathrm{V}}_{\mathrm{new}}$$

(2)

where X and V stand for the current position and velocity of particle, respectively. ${\mathrm{X}}_{\mathrm{new}}$ and ${\mathrm{V}}_{\mathrm{new}}$ represent new position and new velocity of particle, respectively. Moreover, ${\mathrm{C}}_1$ and ${\mathrm{C}}_2$ are two positive acceleration constants with recommended values of 2 for each of them and w is the inertia weight (in the range of 0.25–1). In addition, ${\mathrm{r}}_1$ and ${\mathrm{r}}_2$ denote a random number in a range of (0, 1). The complete or better explanations of PSO details is available in the previous related studies [25,26,28]. The PSO algorithm is shown in Figure 1.

2.3. Hybrid Algorithm

Researchers tried to improve ANN’s efficiency in solving engineering problems by mean of various optimization algorithms such as PSO. Nevertheless, some algorithms like BP that act as a learning algorithm for local search cannot provide acceptable solution for finding ANN optimum solution [61]. However, OAs show satisfactory results in term of regulate bias and weight in ANNs in order to advancement ANN’s prediction. While ANNs show more convergence at local minimums, OAs perform better in global minimum. Therefore, by utilizing ANNs composed with OAs like PSO-ANN (neuro-swarm), one can get advantages of both systems capabilities that means in first step, PSO finds global minimum and then ANN uses their result to optimize solution in local minimum.

The learning procedure of the hybrid neuro-swarm model is displayed in Figure 2. This procedure starts with the initialization of a group of random particles, during which the positions of those particles that are representative of the ANN weights and biases are assigned randomly. After this stage, considering the initial weights and biases (i.e., the initial particle positions), the hybrid neuro-swarm is run and then, the system error is calculated between the predicted and actual values. The calculated error is reduced by changing the positions of particles in each iteration. To update the velocity equation, values of $\overrightarrow{P_{best}}$ (i.e., the lowest error obtained by each particle until that moment) and $\overrightarrow{G_{best}}$ (i.e., the lowest error obtained by all particles until that moment) are utilized. As a result, a value is produced with which to adjust particle positions to the best solutions. Then, a new error is achieved considering the updated positions. Afterwards, the mentioned process continues until the termination criteria are met (which is normally system error).

2.4. Case Study and Data Source

The Klang Valley Mass Rapid Transit (KVMRT) project which was constructed and operated in Kuala Lumpur, Malaysia, was aimed to reduce the traffic crowding in this big city. During site investigation phase of this project, for supporting purposes of the KVMRT project, it was decided to construct many of large-diameter bored piles. Part of the above piles is implemented on different kinds of rocks such as granite and limestone. Our research focuses on outcomes of 96 rock-socketed piles (built in granite rock type). Granitic rock of Triassic era, dominating the undulant terrain, was noticed in the research zone. Subsurface data and materials were accomplished in the pile locations to examine the predominant geological situations. Based on our observations, underlying subsoil contours comprised remaining soils of various rocks (typically granite). In this project, bedrock depth was attained in a range of (0.7–14 m) below the ground level. The overload soil generally contains gravely silt or sandy or silty sand or gravely. Nonetheless, the phrases below summarize borelog-related data and field surveys; (a) a range of slightly weathered rock mass to highly weathered rock mass was observed during the investigations, (b) the minimum and maximum UCS values were obtained as 25 MPa and 68 MPa, respectively (the tests were carried out based on ISRM [62]), (c) till the depth of 16.5 m, a highly weathered residual soil was observed and the SPT N-values of the main soil (hard sandy silt) were obtained in a range of (4–167) blows/300 mm.

The first step for conducting a predictive model is to prepare an appropriate database. Indeed, identification of parameters with the highest influence on model output is considered as a precondition for a predictive intelligence system. As noted above, the results of 96 pile driving analyzer (PDA) tests conducted on rock-socketed piles were taken into consideration for developing hybrid model. The tests were carried out with the help of pile driving analyzer equipment manufactured by Pile Dynamic, Inc. (Cleveland, OH, USA). According to previous investigations, piling geometry factors play a very important role on pile settlement. In this study, relevant parameters i.e., L_s/L_r (length of pile in the soil layer/length of pile in the rock layer) and L_p/D (total length of pile/pile diameter), were selected to be considered as inputs. Additionally, UCS parameter was also selected as a model input because of its importance in the part of pile socketed into the rock. Another input parameter taken into consideration was SPT N-value for representing the situation of soil layer. There is no need to point out that the pile load affects the pile settlement; hence, pile ultimate bearing capacity (Q_u) was fixed as another input to predict the pile settlement (S_p). Ranges of (3.9–31.6), (0.23–32.7), (3.7–166.8), (25–68 MPa), (12,300–43,000 kN) and (5–20 mm), were obtained for L_p/D, L_s/L_r, SPT N-value, UCS, Q_u and S_p, respectively. To have a better understanding on the established database, input and output parameters for all 96 datasets are shown in Figure 3. In the next section, approaching the implement of neuro-swarm system in predicting pile settlement will be described.

3. Model Development

After preparing and establishing the database, settlement of the piles can be predicted in a form of Sp = f [Lp/D, Ls/Lr, SPT N-value, UCS and Qu]. Therefore, this part presents the modeling procedure of ANN and neuro-swarm models in estimating the settlement of rock-socketed piles. In the following, first, an ANN model is designed and an architecture of ANN is introduced for pile settlement prediction. Then, the design stages of the neuro-swarm model will describe in order to show how to solve the problem of this study.

3.1. ANN Design

This section describes the implementation of the ANN model to predict pile settlement. Prior to start ANN modeling for prediction of pile settlement, all datasets regarding training or testing feature must be separated. Nelson and Illingworth [63] based on their studies suggested that (20%–30%) of whole datasets should be considered as testing datasets. Accordingly, we allocated 20% (19 datasets) of whole datasets to testing datasets. Since prosperous applying of Levenberg–Marquardt (LM) training algorithm has noticed in many researches [64,65], it was also used in this study in order to design ANN. As an undeniable fact related to number of hidden layer, according to several researchers [29,66,67], almost all problems can be approximated through the use of only one hidden layer. On the other hand, based on results of the study conducted by Hornik et al. [68], the maximum number of nodes in a hidden layer is 2 $\times$ N_input + 1, where N_input is the number of model inputs. Replacing N_input = 5 in this equation indicates that the problem of pile settlement can be solved by a range of 1 to 11. Therefore, several ANN models with hidden node numbers in the mentioned range were constructed and the results of analyses are shown based on coefficient of determination (R²) in

Table 1. Results of neuro-swarm models with different swarm size (SS) values.

No. of Hidden Neuron		1	2	3	4	5	6	7	8	9	10	11
R2	Train	0.753	0.782	0.809	0.805	0.803	0.784	0.755	0.774	0.799	0.702	0.665
	Test	0.766	0.743	0.816	0.812	0.795	0.757	0.760	0.789	0.802	0.741	0.766

. The formula of R² is presented in follows:

$$R^2=1-\frac{{{\displaystyle \sum }}_i{\left(x_{imeas}-x_{ipred}\right)}^2}{{{\displaystyle \sum }}_i{\left(x_{imeas}-\overline{x}\right)}^2}$$

(3)

where, $x_{imeas}$ and $x_{ipred}$ are the measured and predicted values, respectively. $\overline{x}$ represents the average measured values. According to results of Table 1, an ANN model with only three hidden nodes performs better compared to the others. This model with R² of 0.809 and 0.816 for train and test stages, respectively, shows an acceptable performance prediction in forecasting pile settlement proposing ANN model. Hence, an architecture of ANN with the size of 5 × 3 × 1 was found to offer more enhanced results. Note that, in the suggested ANN architecture, 5 is the number of input parameters, 3 is the number of hidden neurons and 1 is number of output parameter.

3.2. Neuro-Swarm Design

In this part, neuro-swarm modeling procedure to predict pile settlement is presented considering the suggested ANN architecture in previous section. It is worth mentioning that neuro-swarm models were constructed using 80% of the whole database as training datasets and 20% of the whole database as testing datasets. Approaching the implement of neuro-swarm model starts by determining the number of swarm or swarm size (SS). In order to select the best SS, values of 25, 50, 75, 100, 150, 200, 250, 300, 350 and 400 were employed and through a parametric study the best one among them was selected.

Table 2. Results of neuro-swarm models with different SS values.

Model No.	SS	Neuro-Swarm Results				Ranking				Total Rank
		Training		Testing		Training		Testing
		R2	RMSE	R2	RMSE	R2	RMSE	R2	RMSE
1	25	0.749	0.104	0.909	0.098	3	6	10	5	24
2	50	0.772	0.101	0.885	0.107	4	8	9	3	24
3	75	0.828	0.102	0.785	0.087	9	7	3	8	27
4	100	0.805	0.101	0.819	0.109	6	8	6	2	22
5	150	0.818	0.095	0.781	0.123	7	10	2	1	20
6	200	0.827	0.010	0.771	0.103	8	9	1	4	22
7	250	0.827	0.102	0.811	0.076	8	7	5	10	30
8	300	0.796	0.106	0.881	0.081	5	5	8	9	27
9	350	0.844	0.095	0.792	0.096	10	10	4	6	30
10	400	0.828	0.095	0.860	0.089	9	10	7	7	33

tabulates the parametric study’s results. In conducting this parametric study, other effective parameters of PSO were considered constant as follows ($C_1$ = $C_2$ = 2), (max iteration = 100) and (w = 0.25). Ranking system used in Table 2, was utilized according to the study conducted by Zorlu et al. [69]. According to this method, each performance index (i.e., herein root mean square error, RMSE and R²) is ordered in its class. For example, values of 0.749, 0.772, 0.828, 0.805, 0.818, 0.827, 0.827, 0.796, 0.844 and 0.828 were obtained for training datasets of neuro-swarm model numbers of 1–10, with rankings of 3, 4, 9, 6, 7, 8, 8, 5, 10 and 9, respectively. Therefore, as a result, model number 10 with total ranking of 33 was chosen as the best model and 400 was selected as the best SS. Note that, the equation of RMSE is presented as follows:

$$RMSE=\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N {\left(x_{ipred}-x_{imeas}\right)}^2}$$

(4)

where, $x_{imeas}$ and $x_{ipred}$ are the measured and predicted values, respectively. N is the total No. of datasets.

The second parametric study is related to determination of max iteration in predicting pile settlement. The same SS values of the previous stage were used with max iteration number of 500 as shown in Figure 4. The results of this parametric study were assessed based on RMSE values. According to Figure 4, the changes of RMSE values are very small for all SS after iteration number of 400. Hence, the max iteration number was considered as 400.

The next stage of modeling, it seems that there is a need to conduct another parametric study to determine coefficients of velocity equation. Kennedy and Eberhart [58] mentioned that the highest performance prediction of neuro-swarm model can be obtained when $C_1$ = $C_2$ = 2 with total summation of 4. Therefore, another parametric study with various combinations of $C_1$ and $C_2$ (with total summation of 4) was considered and applied in order to choose the best $C_1$ and $C_2$.

Table 3. Neuro-swarm models with different coefficient of velocity equations.

Model No.	C1	C2	Neuro-Swarm Results				Ranking				Total Rank
			Training		Testing		Training		Testing
			R2	RMSE	R2	RMSE	R2	RMSE	R2	RMSE
1	0.8	3.2	0.812	0.092	0.920	0.079	2	6	7	7	22
2	1.333	2.667	0.849	0.092	0.741	0.114	4	6	3	2	15
3	1.714	2.286	0.854	0.092	0.844	0.083	5	6	6	6	23
4	3.2	0.8	0.785	0.109	0.830	0.098	1	4	5	3	13
5	2.667	1.333	0.813	0.104	0.825	0.087	3	5	4	5	17
6	2.286	1.714	0.858	0.089	0.671	0.133	6	7	2	1	16
7	2	2	0.873	0.089	0.670	0.090	7	7	1	4	19

presents the results of these combinations based on R² and RMSE. According to this table and also application of simple ranking by Zorlu et al. [69], model number 3 with the combination of $C_1$ = 1.714 and $C_2$ = 2.286 outperforms the other implemented models. These $C_1$ and $C_2$ values will be used for the rest of neuro-swarm models. It is worth noting that in these models, SS = 400 and max iteration of 400 obtained from the previous stages, were used.

Selection of the appreciate value for inertia weight is the next modeling stage. This has been done by using the different values of w i.e., 0.25, 0.5, 0.75 and 1 to see their prediction performance in estimating pile settlement. In conducting this parametric study, the obtained values from the previous stages have been considered and utilized.

Table 4. Neuro-swarm models with different inertia weights.

Model No.	Inertia Weight	Neuro-Swarm Results				Ranking				Total Rank
		Training		Testing				Training
		R2	RMSE	R2	RMSE	R2	RMSE	R2	RMSE
1	0.25	0.836	0.093	0.867	0.092	2	1	3	3	9
2	0.5	0.848	0.088	0.805	0.114	3	3	1	1	8
3	0.75	0.851	0.087	0.856	0.103	4	4	2	2	12
4	1	0.851	0.089	0.892	0.083	4	2	4	4	14

shows results of 4 neuro-swarm models with w = 0.25, w = 0.5, w = 0.75 and w = 1 based on their performance prediction together with rankings and total rankings of each model. As a result, the highest prediction performance can be achieved when inertia weight set as 1 (model number 4). Hence, w = 1 was chosen as the best inertia weight in modeling of neuro-swarm intelligence system.

As the last modeling stage of neuro-swarm system, there is a need to divide the database into 5 different trains and tests sections to see their effects on system’s results. The process has been suggested in the previous studies as well [19,70]. Five neuro-swarm models have been constructed to forecast settlement of the pile according to the architecture of ANN (5 × 3 × 1), SS = 400, max iteration = 400, $C_1$ = 1.714 and $C_2$ =2.286 and w = 1. Five time running is due to the fact that the system could use the randomly selected datasets, train them and test them to see their effects on the neuro-swarm model performance. The performance predictions of these hybrid models to forecast the pile settlement will be assessed in details later.

4. Results and Discussion

During ANN model development, it was found that a model with three hidden neurons provides the best performance prediction for estimation of pile settlement. The R² values of 0.809 and 0.816 for train and test stages were achieved respectively for the best ANN model. Then, in order to propose higher performance capacity in predicting pile settlement, the effects of the most important PSO parameters on the neuro-swarm system have been investigated. It was observed that by determining of each effective PSO parameter, the prediction performance of neuro-swarm system improves based on system error and R². Finally, based on the obtained values of the most significant PSO parameters, five neuro-swarm models have been built to predict pile settlement. To select the best predictive neuro-swarm model, five performance indices including variance account for (VAF), R², mean absolute error (MAE), RMSE and the a20-index have been used and applied. These performance indices have been widely utilized to assess model performance in the previous related works [23,71,72,73,74,75,76,77,78,79,80,81]. The computation formulas of VAF, MAE and a20-index are presented as follows:

$$VAF=[1-\frac{var (x_{imeas}-x_{ipred})}{var (x_{imeas})} ]\times 100$$

(5)

$$MAE= \frac{1}{N}{\displaystyle \sum }_{i=1}^N\left|x_{ipred}-x_{imeas}\right|$$

(6)

$$a20-index=\frac{m20}{N}$$

(7)

where, $x_{imeas}$ and $x_{ipred}$ are the measured and predicted values, respectively. N is the total No. of datasets, and m20 is the No. of samples with values of rate measured/predicted value (range between 0.8–1.2). The values of 1, 100%, 0 and 0 are required for performance indices of $R^2$, VAF, RMSE and MAE, respectively, when a perfect predictive model is of interest. It is important to mention that the a20-index with value of one should be obtained for a perfect predictive model.

Table 5. Obtained results of performance indices for 5 neuro-swarm hybrid models.

		R2		RMSE		VAF		MAE		a20-index		Total Rank	Final Rank
Model No.	Dataset	Value	Rank	Value	Rank	Value	Rank	Value	Rank	Value	Rank
1	Training	0.884	5	0.082	4	88.306	5	0.066	5	0.675	4	23	29
	Testing	0.623	1	0.156	1	53.748	1	0.130	1	0.421	2	6
2	Training	0.854	2	0.093	2	85.206	2	0.078	1	0.610	2	9	31
	Testing	0.844	4	0.087	5	83.581	4	0.072	5	0.526	4	22
3	Training	0.869	4	0.085	3	86.877	4	0.069	4	0.688	5	20	31
	Testing	0.775	2	0.114	2	77.526	2	0.092	2	0.474	3	11
4	Training	0.851	1	0.079	5	85.056	1	0.071	3	0.649	3	13	32
	Testing	0.892	5	0.099	3	84.738	5	0.087	3	0.474	3	19
5	Training	0.856	3	0.093	2	85.532	3	0.074	2	0.558	1	11	30
	Testing	0.821	3	0.098	4	81.929	3	0.078	4	0.684	5	19

represents the prediction performance of 5 neuro-swam models in estimating settlement of pile. Great performances of the training datasets imply the fruitfulness of the learning process. Moreover, more accurate testing datasets results reveal the well-generalized nature of the proposed models. It should be noted that the authors split data into the training (80%) and testing (20%) datasets in this study. This combination has been suggested and used by many scholars (e.g., [65,82,83]).

Since the best neuro-swarm model in Table 5 was barely recognizable because the results of 5 models are very close to each other, the ranking system proposed by Zorlu et al. [69] has been used to recognize the best model similar to the model development stages. In this table, the total rank refers to the ranking values obtained by each datasets (i.e., training and testing), while, the final rank is a summation value of total rank for the specific datasets (train and test). Therefore, the final rank values should be used for comparison purposes of the neuro-swarm models. The best results in predicting pile settlement are related to model number 4 with final rank of 32 and total rank values of 13 and 19, respectively for train and test. Results of (0.851, 0.079, 85.056%, 0.071 and 0.649) and (0.892, 0.099, 84.738%, 0.087 and 0.474) were obtained for the performance indices of R², RMSE, VAF, MAE and a20-index, respectively. Generally, these results reveal that the neuro-swarm model is capable enough to predict pile settlement. Figure 5 and Figure 6 depict the predicted pile settlement values along with their actual values for train and test sections of the best neuro-swarm model, respectively. These two figures show the ability of the neuro-swarm model in solving pile settlement with low level of system error.

It is important to say that the same database has been applied by Armaghani et al. [3] to introduce GEP model for pile settlement estimation. In that study, they successfully indicated that GEP model is able to deliver higher capacity in comparison with the MLR model. The results of GEP model based on R² were 0.872 and 0.861, respectively for train and test stages. Considering results of the present study, it was found that the selected neuro-swarm model is able to perform better in testing or model evaluation section compared to GEP model by Armaghani et al. [3]. This shows that neuro-swarm model is a strong technique in part of model development. On the other hand, the evaluation in the present study is based on 5 performance indices compared to the study by Armaghani et al. where 3 performance indices have been used for system evaluation. Needless to say that evaluation process of predictive techniques with more details is always of interest and advantage. In conclusion of this part, the developed neuro-swarm model is a powerful hybrid intelligence system that can solve ANN shortcomings and it can be applied for similar purpose of this study in other fields of engineering.

5. Conclusions

To implement a hybrid ANN model namely neuro-swarm, results of 96 PDA tests together with properties of piles and earth were considered and used. In this respect, five parameters, including L_s/L_r, L_p/D, UCS, SPT N-value and Q_u were set as inputs to forecast S_p. In fact, through developing neuro-swarm model, we intend to have the best relationship of S_p = f (L_s/L_r, L_p/D, UCS, SPT N-value and Q_u). According to the most effective PSO parameters, several parametric studies have been conducted to first select the best one among them and then improve the prediction performance of neuro-swarm model in predicting pile settlement. It was found that the prediction performance of neuro-swarm is improved by fixing the effective PSO parameters in each stage. At the last modeling stage, 5 neuro-swarm models were constructed to see effects of various combinations of training and testing datasets. Then, these 5 models in estimating pile settlement were evaluated using five performance indices i.e., R², RMSE, VAF, MAE and a20-index. Eventually, a slightly higher prediction of performance was recorded by dataset number 4 among all 5 neuro-swarm models. Based on R², and system RMSE, values of (0.851 and 0.079) and (0.892 and 0.099) were observed for train and test stages, respectively, which confirm the high strength of neuro-swarm model in predicting pile settlement. It should be noted that the results of a pre-developed ANN model in predicting pile settlement based on R² were obtained as 0.809 and 0.816 for training and testing datasets, respectively, which showed a lower performance capacity of ANN model compared to neuro-swarm predictive technique. Therefore, by developing the neuro-swarm model in predicting pile settlement, the performance capacity based on R² can be significantly improved i.e., from 0.809 to 0.851 for training datasets and from 0.816 to 0.892 for testing datasets. To conclude, the neuro-swarm model can be successfully applied in order to solve ANN shortcomings. Approaching of the implement of neuro-swarm technique presented in this study can be used by other researchers for solving problems in fields of engineering.