Design of Aquila Optimization Heuristic for Identification of Control Autoregressive Systems

Abstract

Swarm intelligence-based metaheuristic algorithms have attracted the attention of the research community and have been exploited for effectively solving different optimization problems of engineering, science, and technology. This paper considers the parameter estimation of the control autoregressive (CAR) model by applying a novel swarm intelligence-based optimization algorithm called the Aquila optimizer (AO). The parameter tuning of AO is performed statistically on different generations and population sizes. The performance of the AO is investigated statistically in various noise levels for the parameters with the best tuning. The robustness and reliability of the AO are carefully examined under various scenarios for CAR identification. The experimental results indicate that the AO is accurate, convergent, and robust for parameter estimation of CAR systems. The comparison of the AO heuristics with recent state of the art counterparts through nonparametric statistical tests established the efficacy of the proposed scheme for CAR estimation.

1. Introduction

1.1. Literature Review

In recent years, system identification has gained significant attention in various areas such as signal processing, parameter estimation, multiple-input multiple-output systems, etc. [1,2,3,4]. Parameter estimation refers to the determination of the best fitness values for each parameter using local or global optimization techniques. Parameter estimation consists of three steps: First, construct a mathematical model for a given system such that it replicates exact behaviour under the same conditions. Second, define a fitness function for a given set of parameters using various approximations such as least square, weighted least square, and generalized least square. Third, select an optimization technique for finding the best fitness values through iteration [5].

The research community has shown great interest in parameter estimation of control autoregressive (CAR) systems because of their importance and significance in effectively modelling a variety of engineering problems including power system optimization [6] electricity load prediction [7] battery charge estimation [8] forecasting groundwater flooding [9] and CO₂ emission forecasting [10]. Various methods for parameter estimation of control autoregressive (CAR) models are proposed in the literature. Ding et al. [11] decompose a CAR model into two subsystems and derive a two-stage multi-innovation gradient-based iterative algorithm for parameter estimation. Raja et al. [12] use genetic algorithms (GA) for parameter estimation of a nonlinear Hammerstein controlled autoregressive system by minimizing the error function between true and estimated parameters. Mehmood et al. [13] explore the strengths of evolutionary and swarm intelligence for parameter estimation of a controlled autoregressive moving average model by minimizing mean absolute error and other measures for parameter estimation. Tariq et al. [14] apply differential evolution-based algorithms for parameter estimation of Hammerstein systems minimizing the actual and predicted responses of cost function output errors.

Metaheuristic methods have made significant progress in solving optimization problems [15,16,17]. These optimization methods can be classified into four categories. Category one includes evolutionary algorithms, which involve natural biological behaviour such as mutation and crossover operation. Numerous algorithms are proposed in this category, such as genetic algorithms, GAs [18] differential evolution, DE [19] fuzzy evolution [20] maximum likelihood adaptive differential evolution [21] and tree growth algorithms [22]. Category two includes human-based algorithms, which are inspired from human behaviour such as collective decision optimization [23] imperialist competitive algorithms [24] and teaching-learning-based optimization [25]. Category three includes physics-based methods which use physical laws for solution of optimization problems. A few of the methods in this area are gravitational search algorithms [26] thermal exchange optimization [27] and multiverse optimizers [28]. The final category includes swarm intelligence and animal-inspired methods for optimization solution. Significant progress has made in this domain as well, and various methods are proposed such as particle swarm optimization (PSO) [29] artificial bee colonies [30] lion optimization algorithms [31] and whale optimization algorithms [32].

Among swarm intelligence techniques, the Aquila optimizer (AO) [33] has been recently proposed for global solutions and applied in solving various real-world optimization problems. Elaziz et al. [34] proposed an image classification hybrid framework for COVID-19 images by combining deep learning and AO for feature selection and dimensionality reduction for CT and X-ray images. Hussan et al. [35] applied AO in harmonic parameter estimation for an H-bridge inverter by minimizing error with real-time verification on digital signal processing launchpad. Khamees et al. [36] applied AO in Weibull distribution parameter estimation for low error and high correlation coefficients in a wind energy system.

1.2. Research Contribution

In the current study, the swarm intelligence of the Aquila optimizer, AO, is exploited for parameter estimation of control autoregressive (CAR) systems. The AO is evaluated in terms of robustness, correctness, and convergence for different noise levels in the CAR model. The noticeable contributions are as follows:

• The strength of a swarm intelligence-based Aquila optimizer (AO) heuristic is exploited for solving parameter estimation in a control autoregressive (CAR) model.
• The convergence, accuracy, and robustness analyses of the AO are conducted for different noise levels considered in the CAR model.
• Statistical analyses for parameter tuning of the AO as well as for reliability and stability assessment are conducted for different generations and population sizes.

1.3. Paper Organization

The rest of the paper is prepared as follows: the CAR system model is given in Section 2. The AO-based methodology is presented in Section 3. The performance analysis of the CAR model is provided in Section 4. The main conclusions and some future research directions are listed in Section 5.

2. Mathematical Model of CAR Systems

Consider the second-order CAR model presented in (1):

$$\mathrm{R}\left(\mathrm{z}\right)\mathrm{q}\left(\mathrm{t}\right)=\mathrm{S}\left(\mathrm{z}\right)\mathsf{\theta }\left(\mathrm{t}\right)+\mathsf{\omega }\left(\mathrm{t}\right),$$

(1)

where θ(t) is the input of the model, q(t) is the output of the model, and ω(t) is zero mean white noise. R(z) and S(z) are polynomials given in (2) and (3):

$$\mathrm{R}\left(\mathrm{z}\right)=1+{\mathrm{r}}_1{\mathrm{z}}^{-1}+{\mathrm{r}}_2{\mathrm{z}}^{-2}+\dots +{\mathrm{r}}_{{\mathrm{n}}_{\mathrm{r}}}{\mathrm{z}}^{-{\mathrm{n}}_{\mathrm{r}}},$$

(2)

$$\mathrm{S}\left(\mathrm{z}\right)={\mathrm{s}}_1{\mathrm{z}}^{-1}+{\mathrm{s}}_2{\mathrm{z}}^{-2}+\dots +{\mathrm{s}}_{{\mathrm{n}}_{\mathrm{s}}}{\mathrm{z}}^{-{\mathrm{n}}_{\mathrm{s}}}.$$

(3)

Assume that $\mathsf{\theta }\left(\mathrm{t}\right)=0$, $\mathrm{q}\left(\mathrm{t}\right)=0$, $\mathsf{\omega }\left(\mathrm{t}\right)=0$ for t < 0, and ${\mathrm{n}}_{\mathrm{r}}$ and ${\mathrm{n}}_{\mathrm{s}}$ are known. The parameter vectors are given in (4) and (5):

$$\mathrm{r}={\left[{\mathrm{r}}_1,{\mathrm{r}}_2,\dots ,{\mathrm{r}}_{{\mathrm{n}}_{\mathrm{r}}}\right]}^{\mathrm{T}}\in {\mathrm{R}}^{{\mathrm{n}}_{\mathrm{r}}},$$

(4)

$$\mathrm{s}={\left[{\mathrm{s}}_1,{\mathrm{s}}_2,\dots ,{\mathrm{s}}_{{\mathrm{n}}_{\mathrm{s}}}\right]}^{\mathrm{T}}\in {\mathrm{R}}^{{\mathrm{n}}_{\mathrm{s}}}.$$

(5)

The corresponding information vectors are given in (6) and (7):

$${\mathsf{ϵ}}_{\mathrm{r}}\left(\mathrm{t}\right)={\left[-\mathrm{q}\left(\mathrm{t}-1\right),-\mathrm{q}\left(\mathrm{t}-2\right),\dots ,-\mathrm{q}\left(\mathrm{t}-{\mathrm{n}}_{\mathrm{r}}\right)\right]}^{\mathrm{T}}\in {\mathrm{R}}^{{\mathrm{n}}_{\mathrm{r}}},$$

(6)

$${\mathsf{ϵ}}_{\mathrm{s}}\left(\mathrm{t}\right)={\left[\mathsf{\theta }\left(\mathrm{t}-1\right),\mathsf{\theta }\left(\mathrm{t}-2\right),\dots ,\mathsf{\theta }\left(\mathrm{t}-{\mathrm{n}}_{\mathrm{s}}\right)\right]}^{\mathrm{T}}\in {\mathrm{R}}^{{\mathrm{n}}_{\mathrm{s}}}.$$

(7)

The model presented in (1) can be rewritten as given in (8) and simplified in (9).

$$\mathrm{q}\left(\mathrm{t}\right)=\left[1-\mathrm{R}\left(\mathrm{z}\right)\right]\mathrm{q}\left(\mathrm{t}\right)+\mathrm{S}\left(\mathrm{z}\right) \mathsf{\theta }\left(\mathrm{t}\right)+\mathsf{\omega }\left(\mathrm{t}\right),$$

(8)

$$\mathrm{q}\left(\mathrm{t}\right)={\mathsf{ϵ}}_{\mathrm{r}}\left(\mathrm{t}\right)\mathrm{r}+{\mathsf{ϵ}}_{\mathrm{s}}\left(\mathrm{t}\right)\mathrm{s}+ \mathsf{\omega }\left(\mathrm{t}\right).$$

(9)

The overall information and parameter vectors of the CAR model are given as

$$\mathsf{ϵ}\left(\mathrm{t}\right)=[{\mathsf{ϵ}}_{\mathrm{r}}\left(\mathrm{t}\right) {\mathsf{ϵ}}_{\mathrm{s}}\left(\mathrm{t}\right)]$$

(10)

$$\mathsf{\alpha }=\left[\mathrm{r} \mathrm{s}\right].$$

(11)

Then, the identification for CAR system becomes:

$$\mathrm{q}\left(\mathrm{t}\right)={\mathsf{ϵ}}^{\mathrm{T}}\left(\mathrm{t}\right)\mathsf{\alpha }+\mathsf{\omega }\left(\mathrm{t}\right).$$

(12)

3. Methodology

In this section, an AO-based methodology for parameter estimation of a CAR model is presented. The graphical abstract of the proposed methodology for CAR model is presented in Figure 1. It provides the overview of the proposed study, which includes the parameter estimation of a CAR model by applying a swarm intelligence-based Aquila optimizer. The optimum parameters are evaluated on the basis of the square of the difference between estimated and true values along with the number of generations as termination criteria.

3.1. Aquila Optimization (AO) Method

The AO is a swarm intelligence-based method for finding solutions to optimum global problems [33]. It is applied in various domains such as internet of things (IoT) [37], power electronics [35], image processing [38], oil production forecasting [39], Francis turbines [40], hybrid solid oxide fuel cell (SOFC) [41], wind energy [42], and population forecasting [43]. AO is a population-based optimization method inspired by Aquila’s prey-hunting ability. It uses four hunting techniques and has the ability to switch between these techniques. The mathematical model, pseudocode, and flowchart are presented below.

3.1.1. Population Initialization

AO starts with the initialization of the population for candidate solutions (W) as given in (13):

$$\mathrm{W}=\left[\begin{array}{ccc}{\mathrm{w}}_{1,1}& \cdots & {\mathrm{w}}_{1,\mathrm{D}}\\ ⋮& \ddots & ⋮\\ {\mathrm{w}}_{{\mathrm{N}}_{\mathrm{p}},1}& \cdots & {\mathrm{w}}_{{\mathrm{N}}_{\mathrm{p}},\mathrm{D}}\end{array}\right].$$

(13)

The population is randomly generated using (14):

$${\mathrm{W}}_{\mathrm{k},\mathrm{l}}=\mathrm{rand}\left({\mathrm{UB}}_{\mathrm{l}}-{\mathrm{LB}}_{\mathrm{l}}\right)+{\mathrm{LB}}_{\mathrm{l}}, \mathrm{k}=1,2\dots {\mathrm{N}}_{\mathrm{p}},\mathrm{l}=1,2\dots \mathrm{D}$$

(14)

where ${\mathrm{N}}_{\mathrm{p}}$ is the total population size, and $\mathrm{D}$ is the number of decision variables.

3.1.2. The Mathematical Model

The mathematical formulation is divided into four steps which are presented below.

Expanded Exploration (${\mathrm{W}}_1$)

In the first method, ${\mathrm{W}}_1$, the Aquila explores the prey area by involving a high soar and a vertical stoop. It is presented in (15):

$${\mathrm{W}}_1\left(\mathrm{t}+1\right)={\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)\times \left(1-\frac{\mathrm{t}}{\mathrm{T}}\right)+\left({\mathrm{W}}_{\mathrm{M}}\left(\mathrm{t}\right)-{\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)\ast \mathrm{rand}\right)$$

(15)

where ${\mathrm{W}}_1\left(\mathrm{t}+1\right)$ is the next-iteration solution for ${\mathrm{W}}_1$, ${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)$ is the best solution, $\left(1-\frac{\mathrm{t}}{\mathrm{T}}\right)$ is used to control the search space, and ${\mathrm{W}}_{\mathrm{M}}\left(\mathrm{t}\right)$ is the mean of the current solution, which is calculated using (16):

$${\mathrm{W}}_{\mathrm{M}}\left(\mathrm{t}\right)=\frac{1}{{\mathrm{N}}_{\mathrm{p}}}{\displaystyle \sum }_{\mathrm{k}=1}^{{\mathrm{N}}_{\mathrm{p}}}{\mathrm{W}}_{\mathrm{k}}\left(\mathrm{t}\right),\forall \mathrm{l}=1,2\dots \mathrm{D}$$

(16)

where ${\mathrm{N}}_{\mathrm{p}}$ is the total population size, and $\mathrm{D}$ is the number of decision variables.

Narrowed Exploration (${\mathrm{W}}_2$)

In the second method (${\mathrm{W}}_2$), upon finding the prey area, the Aquila circles around the target and uses a method called contour fight with a short glide attack. In ${\mathrm{W}}_2$, AO narrowly explores the area for preparation of the attack on the target, which is calculated using (17):

$${\mathrm{W}}_2\left(\mathrm{t}+1\right)={\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)\times \mathrm{LEF}\left(\mathrm{DI}\right)+{\mathrm{W}}_{\mathrm{R}}\left(\mathrm{t}\right)-\left(\mathrm{y}-\mathrm{w}\right)\ast \mathrm{rand}$$

(17)

where ${\mathrm{W}}_2\left(\mathrm{t}+1\right)$ is the next-iteration solution for ${\mathrm{W}}_2$, $\mathrm{DI}$ is the space dimension, ${\mathrm{W}}_{\mathrm{R}}\left(\mathrm{t}\right)$ is the random solution from [1 ${\mathrm{N}}_{\mathrm{p}}$], and $\mathrm{LEF}\left(\mathrm{DI}\right)$ is the distribution function calculated in (18):

$$\mathrm{LEF}\left(\mathrm{DI}\right)=\mathrm{d}\times \frac{\mathrm{e}\times \mathsf{\sigma }}{{\left|\mathrm{f}\right|}^{\frac{1}{\mathsf{\delta }}}}$$

(18)

where $\mathrm{d}$ is constant equals 0.01,$\mathrm{e}$ and $\mathrm{f}$ are random numbers between 0–1.$\mathsf{\sigma }$ is calculated using (19):

$$\mathsf{\sigma }=\left(\frac{Ґ\left(1+\mathsf{\delta }\right)\times \sin \left(\frac{\mathsf{\pi }\mathsf{\delta }}{2}\right)}{Ґ\left(\frac{1+\mathsf{\delta }}{2}\right)\times \mathsf{\delta }\times 2^{\left(\frac{\mathsf{\delta }-1}{2}\right)}}\right)$$

(19)

where $\mathsf{\delta }$ is fixed at 1.5. $\mathrm{y}$ and $\mathrm{w}$ from (17) are calculated as follows.

$$\mathrm{y}=\mathrm{g}\times \cos \left(\mathsf{\theta }\right),$$

(20)

$$\mathrm{w}=\mathrm{g}\times \sin \left(\mathsf{\theta }\right)$$

(21)

where $\mathrm{g}$ is between 1–20.

$$\mathrm{g}={\mathrm{g}}_1+\mathrm{V}\times {\mathrm{DI}}_1,$$

(22)

$$\mathsf{\theta }=-\mathsf{\epsilon }\times {\mathrm{DI}}_1+{\mathsf{\theta }}_1,$$

(23)

$${\mathrm{\theta }}_1=\frac{3\mathrm{\pi }}{2}$$

(24)

where $\mathrm{V}$ is fixed to 0.00565 and $\mathsf{\epsilon }$ is fixed to 0.005.

Expanded Exploitation (${\mathrm{W}}_3$)

In the third method (${\mathrm{W}}_3$), AO exploits the search space by descending vertically to discover prey reaction and to land and attack. It is given in (25):

$${\mathrm{W}}_3\left(\mathrm{t}+1\right)=\left({\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)-{\mathrm{W}}_{\mathrm{M}}\left(\mathrm{t}\right)\right)\times \mathsf{\beta }-\mathrm{rand}+\left(\left(\mathrm{UB}-\mathrm{LB}\right)\times \mathrm{rand}+\mathrm{LB}\right)\times \mathsf{\mu }$$

(25)

where ${\mathrm{W}}_3\left(\mathrm{t}+1\right)$ is the next-iteration solution for ${\mathrm{W}}_3$,$\mathsf{\beta }$ and $\mathsf{\mu }$ are exploitation adjustment factors,${ \mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)$ is the best solution, $\mathrm{UB}$ and $\mathrm{LB}$ are problem-dependent parameters, and ${\mathrm{W}}_{\mathrm{M}}\left(\mathrm{t}\right)$ is the mean of the current solution, which is calculated using (16).

Narrowed Exploitation (${\mathrm{W}}_4$)

In the fourth method (${\mathrm{W}}_4$), AO uses the method of walking and grabbing by getting closer to prey and attacking based on stochastic movement as presented in (26):

$${\mathrm{W}}_4\left(\mathrm{t}+1\right)=\mathrm{QYF}\times {\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)-\left({\mathrm{O}}_1\times \mathrm{W}\left(\mathrm{t}\right)\times \mathrm{rand}\right)-{\mathrm{O}}_2\times \mathrm{LEF}\left(\mathrm{DI}\right)+\mathrm{rand}\times {\mathrm{O}}_1$$

(26)

where ${\mathrm{W}}_4\left(\mathrm{t}+1\right)$ is the next-iteration solution for ${\mathrm{W}}_4$ and $\mathrm{QYF}$ is the quality factor, calculated using (27):

$$\mathrm{QYF}={\mathrm{t}}^{\frac{2\mathrm{rand}-1}{{\left(1-\mathrm{T}\right)}^2}}$$

(27)

${\mathrm{O}}_1$ and ${\mathrm{O}}_2$ indicate the variations of motion, which are calculated using (28) and (29):

$${\mathrm{O}}_1=2\times \mathrm{rand}-1,$$

(28)

$${\mathrm{O}}_2=2\times \left(1-\frac{\mathrm{t}}{\mathrm{T}}\right)$$

(29)

The flowchart for AO is shown in Figure 2.

The pseudocode of the MLADE is presented in Algorithm 1.

Algorithm 1: Pseudo-code of AO

Initialization:

Initialize the population W and parameters of AO such as σ, β, etc.

WHILE do

Calculate fitness values

Determine the best obtained solution ${\mathrm{W}}_{\mathrm{best}}$.

for$\mathrm{k}=1:{ \mathrm{N}}_{\mathrm{p}}$

$\mathrm{Update} \mathrm{mean} \mathrm{value} {\mathrm{W}}_{\mathrm{M}}\left(\mathrm{t}\right)$.

$\mathrm{Update} \mathrm{w},\mathrm{y},{\mathrm{O}}_1$,${\mathrm{O}}_2$ and $\mathrm{LEF}\left(\mathrm{DI}\right)$.

if $\mathrm{t}\le \left(\frac{2}{3}\right)\ast \mathrm{T}$

if $\mathrm{rand}\le 0.5$

Expanded Exploration ($W_1$)

Update solution using (15).

If (Fitness ${\mathrm{W}}_1\left(\mathrm{t}+1\right)<$ Fitness $\mathrm{W}\left(\mathrm{t}\right)$)

$\mathrm{W}\left(\mathrm{t}\right)={\mathrm{W}}_1\left(\mathrm{t}+1\right)$

If (Fitness ${\mathrm{W}}_1\left(\mathrm{t}+1\right)<$ Fitness ${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)$)

${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)={\mathrm{W}}_1\left(\mathrm{t}+1\right)$

end

end

else

Narrowed Exploration ($W_2$)

Update solution using (17).

If (Fitness ${\mathrm{W}}_2\left(\mathrm{t}+1\right)<$ Fitness $\mathrm{W}\left(\mathrm{t}\right)$)

$\mathrm{W}\left(\mathrm{t}\right)={\mathrm{W}}_2\left(\mathrm{t}+1\right)$

If (Fitness ${\mathrm{W}}_2\left(\mathrm{t}+1\right)<$ Fitness ${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)$)

${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)={\mathrm{W}}_2\left(\mathrm{t}+1\right)$

end

end

end

else if $\mathrm{rand}\le 0.5$

Expanded Exploitation ($W_3$)

Update solution using (25).

If (Fitness ${\mathrm{W}}_3\left(\mathrm{t}+1\right)<$ Fitness $\mathrm{W}\left(\mathrm{t}\right)$)

$\mathrm{W}\left(\mathrm{t}\right)={\mathrm{W}}_3\left(\mathrm{t}+1\right)$

If (Fitness ${\mathrm{W}}_3\left(\mathrm{t}+1\right)<$ Fitness ${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)$)

${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)={\mathrm{W}}_3\left(\mathrm{t}+1\right)$

end

end

else

Narrowed Exploitation ($W_4$)

Update solution using (26).

If (Fitness ${\mathrm{W}}_4\left(\mathrm{t}+1\right)<$ Fitness $\mathrm{W}\left(\mathrm{t}\right)$)

$\mathrm{W}\left(\mathrm{t}\right)={\mathrm{W}}_4\left(\mathrm{t}+1\right)$

If (Fitness ${\mathrm{W}}_4\left(\mathrm{t}+1\right)<$ Fitness ${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)$)

${\mathrm{W}}_{\mathrm{best}}\left(\mathrm{t}\right)={\mathrm{W}}_4\left(\mathrm{t}+1\right)$

end

end

end

end

end

end

return ${\mathrm{W}}_{\mathrm{best}}$

4. Performance Analysis

In this section, the performance analysis of AO for the CAR model is presented. The identification of the CAR model is conducted on various noise levels, generations, and population sizes. The algorithm is weighed in terms of accuracy, which is measured by the fitness function presented in (30):

$$\mathrm{Fitness} \mathrm{Function}=\mathrm{mean}{\left(z-\widehat{z}\right)}^2$$

(30)

where $\hat{\mathrm{z}}$ is the estimated response through the proposed evolutionary algorithms and $\mathrm{z}$ is the desired response. For the simulation study, we considered the second-order CAR model from [6] as presented in (31) and (32):

$$\mathrm{R}\left(\mathrm{z}\right)=1+1.35{\mathrm{z}}^{-1}+0.75{\mathrm{z}}^{-2},$$

(31)

$$\mathrm{S}\left(\mathrm{z}\right)=1.68{\mathrm{z}}^{-1}+2.32{\mathrm{z}}^{-2}.$$

(32)

4.1. Parameter Tuning of AO

Learning optimal weights plays a significant role in boosting the performance of the AO method. Hence, the aim is to use the best values for the exploitation adjustment factors ($\mathsf{\beta }$ and $\mathsf{\mu }$) for learning the optimum weights ${\mathrm{W}}_3$ using the update rule given in (25). The best values for both parameters ($\mathsf{\beta }$ and $\mathsf{\mu }$) are achieved through hyper-parameter tuning. Using grid search, various combinations of both $\mathsf{\beta }$ and $\mathsf{\mu }$ are split into different cases like case 1 to case 9, and the chosen values of $\mathsf{\beta }$ and $\mathsf{\mu }$ are presented in

Table 1. Parameter-tuning cases for AO analysis.

Case No.	β Value	μ Value
1	0.9	0.9
2	0.9	0.5
3	0.9	0.1
4	0.5	0.9
5	0.5	0.5
6	0.5	0.1
7	0.1	0.9
8	0.1	0.5
9	0.1	0.1

. Hyper-parameter tuning is performed for different generations and population sizes in a noise-free environment (zero noise). Each case is executed for three generations, i.e., 1000, 1500, and 2000, and populations, i.e., 30, 40, and 50, whereas the simulations for a combination of one generation and one population are performed for 15 runs to accomplish the average fit, best fit, worst fit, and standard deviation.

Different scenarios (cases) reflecting the tuning of $\mathsf{\beta }$ and $\mathsf{\mu }$ for the optimal weight update mechanism and the average fit, best fit, worst fit, and standard deviation with different generations and populations are computed and represented in

Table 2. AO parameter analysis for case 1.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$1.2\times {10}^{-3}$	$1.2\times {10}^{-4}$	$3.3\times {10}^{-3}$	$8.7\times {10}^{-4}$
	40	$1.3\times {10}^{-3}$	$1.8\times {10}^{-4}$	$4.0\times {10}^{-3}$	$1.0\times {10}^{-3}$
	50	$6.9\times {10}^{-4}$	$1.4\times {10}^{-4}$	$2.4\times {10}^{-3}$	$6.1\times {10}^{-4}$
1500	30	$1.5\times {10}^{-3}$	$4.1\times {10}^{-4}$	$4.1\times {10}^{-3}$	$1.2\times {10}^{-3}$
	40	$8.8\times {10}^{-4}$	$6.2\times {10}^{-5}$	$2.2\times {10}^{-3}$	$6.7\times {10}^{-4}$
	50	$7.0\times {10}^{-4}$	$9.0\times {10}^{-5}$	$1.6\times {10}^{-3}$	$4.7\times {10}^{-4}$
2000	30	$6.4\times {10}^{-4}$	$1.5\times {10}^{-4}$	$2.0\times {10}^{-3}$	$4.9\times {10}^{-4}$
	40	$7.7\times {10}^{-4}$	$1.7\times {10}^{-4}$	$2.5\times {10}^{-3}$	$6.4\times {10}^{-4}$
	50	$4.5\times {10}^{-4}$	$1.2\times {10}^{-4}$	$1.1\times {10}^{-3}$	$3.1\times {10}^{-4}$

,

Table 3. AO parameter analysis for case 2.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$4.4\times {10}^{-3}$	$3.9\times {10}^{-4}$	$12\times {10}^{-3}$	$3.5\times {10}^{-3}$
	40	$2.6\times {10}^{-3}$	$4.3\times {10}^{-5}$	$16\times {10}^{-3}$	$3.9\times {10}^{-3}$
	50	$2.8\times {10}^{-3}$	$8.9\times {10}^{-5}$	$11\times {10}^{-3}$	$2.9\times {10}^{-3}$
1500	30	$2.9\times {10}^{-3}$	$1.3\times {10}^{-4}$	$9.0\times {10}^{-3}$	$3.0\times {10}^{-3}$
	40	$1.1\times {10}^{-3}$	$1.9\times {10}^{-5}$	$2.7\times {10}^{-3}$	$8.3\times {10}^{-4}$
	50	$1.1\times {10}^{-3}$	$1.2\times {10}^{-4}$	$3.1\times {10}^{-3}$	$7.9\times {10}^{-4}$
2000	30	$1.0\times {10}^{-3}$	$1.9\times {10}^{-4}$	$2.8\times {10}^{-3}$	$7.4\times {10}^{-4}$
	40	$1.1\times {10}^{-3}$	$3.3\times {10}^{-4}$	$3.1\times {10}^{-3}$	$7.3\times {10}^{-4}$
	50	$1.6\times {10}^{-3}$	$2.2\times {10}^{-4}$	$6.5\times {10}^{-3}$	$1.7\times {10}^{-3}$

,

Table 4. AO parameter analysis for case 3.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$1.9\times {10}^{-3}$	$1.4\times {10}^{-4}$	$5.3\times {10}^{-3}$	$1.5\times {10}^{-3}$
	40	$1.4\times {10}^{-3}$	$1.5\times {10}^{-4}$	$3.9\times {10}^{-3}$	$1.2\times {10}^{-3}$
	50	$6.9\times {10}^{-4}$	$4.1\times {10}^{-5}$	$1.8\times {10}^{-3}$	$4.9\times {10}^{-4}$
1500	30	$1.0\times {10}^{-3}$	$1.0\times {10}^{-4}$	$5.2\times {10}^{-3}$	$1.3\times {10}^{-3}$
	40	$8.5\times {10}^{-4}$	$1.7\times {10}^{-4}$	$2.5\times {10}^{-3}$	$6.7\times {10}^{-4}$
	50	$6.4\times {10}^{-4}$	$1.1\times {10}^{-4}$	$1.8\times {10}^{-3}$	$5.2\times {10}^{-4}$
2000	30	$8.5\times {10}^{-4}$	$1.2\times {10}^{-4}$	$2.2\times {10}^{-3}$	$5.6\times {10}^{-4}$
	40	$5.2\times {10}^{-4}$	$3.0\times {10}^{-5}$	$1.0\times {10}^{-3}$	$3.3\times {10}^{-4}$
	50	$5.6\times {10}^{-4}$	$8.2\times {10}^{-5}$	$2.4\times {10}^{-3}$	$5.7\times {10}^{-4}$

,

Table 5. AO parameter analysis for case 4.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$4.4\times {10}^{-3}$	$3.9\times {10}^{-4}$	$12\times {10}^{-3}$	$3.5\times {10}^{-3}$
	40	$2.6\times {10}^{-3}$	$4.3\times {10}^{-4}$	$16\times {10}^{-3}$	$3.9\times {10}^{-3}$
	50	$2.8\times {10}^{-3}$	$8.9\times {10}^{-5}$	$11.5\times {10}^{-3}$	$2.9\times {10}^{-3}$
1500	30	$2.9\times {10}^{-3}$	$1.3\times {10}^{-4}$	$9.0\times {10}^{-3}$	$3.0\times {10}^{-3}$
	40	$1.1\times {10}^{-3}$	$1.9\times {10}^{-5}$	$2.7\times {10}^{-3}$	$8.3\times {10}^{-4}$
	50	$1.1\times {10}^{-3}$	$1.3\times {10}^{-4}$	$3.1\times {10}^{-3}$	$7.9\times {10}^{-4}$
2000	30	$1.0\times {10}^{-3}$	$1.9\times {10}^{-4}$	$2.8\times {10}^{-3}$	$7.4\times {10}^{-4}$
	40	$1.1\times {10}^{-4}$	$3.3\times {10}^{-4}$	$3.1\times {10}^{-3}$	$7.3\times {10}^{-4}$
	50	$1.6\times {10}^{-3}$	$2.2\times {10}^{-4}$	$6.5\times {10}^{-3}$	$1.7\times {10}^{-3}$

,

Table 6. AO parameter analysis for case 5.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$1.1\times {10}^{-3}$	$1.7\times {10}^{-4}$	$3.4\times {10}^{-3}$	$8.9\times {10}^{-4}$
	40	$5.4\times {10}^{-4}$	$7.2\times {10}^{-5}$	$1.4\times {10}^{-3}$	$3.6\times {10}^{-4}$
	50	$5.7\times {10}^{-4}$	$2.3\times {10}^{-5}$	$1.2\times {10}^{-3}$	$2.9\times {10}^{-4}$
1500	30	$6.8\times {10}^{-4}$	$6.7\times {10}^{-5}$	$1.7\times {10}^{-3}$	$5.5\times {10}^{-4}$
	40	$2.2\times {10}^{-4}$	$7.4\times {10}^{-6}$	$6.6\times {10}^{-4}$	$2.1\times {10}^{-4}$
	50	$3.7\times {10}^{-4}$	$1.3\times {10}^{-4}$	$1.2\times {10}^{-3}$	$3.3\times {10}^{-4}$
2000	30	$3.2\times {10}^{-4}$	$3.9\times {10}^{-5}$	$9.6\times {10}^{-4}$	$3.1\times {10}^{-4}$
	40	$2.6\times {10}^{-4}$	$3.1\times {10}^{-5}$	$6.1\times {10}^{-4}$	$1.8\times {10}^{-4}$
	50	$2.1\times {10}^{-4}$	$2.1\times {10}^{-5}$	$5.6\times {10}^{-4}$	$1.5\times {10}^{-4}$

,

Table 7. AO parameter analysis for case 6.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$1.2\times {10}^{-3}$	$1.2\times {10}^{-4}$	$3.3\times {10}^{-3}$	$8.7\times {10}^{-4}$
	40	$1.3\times {10}^{-3}$	$1.8\times {10}^{-4}$	$4.0\times {10}^{-3}$	$1.0\times {10}^{-3}$
	50	$6.9\times {10}^{-4}$	$1.4\times {10}^{-4}$	$2.4\times {10}^{-3}$	$6.1\times {10}^{-4}$
1500	30	$1.5\times {10}^{-3}$	$4.1\times {10}^{-4}$	$4.1\times {10}^{-3}$	$1.2\times {10}^{-3}$
	40	$8.8\times {10}^{-4}$	$6.2\times {10}^{-5}$	$2.2\times {10}^{-3}$	$6.7\times {10}^{-4}$
	50	$7.0\times {10}^{-4}$	$9.0\times {10}^{-5}$	$1.6\times {10}^{-3}$	$4.7\times {10}^{-4}$
2000	30	$6.4\times {10}^{-4}$	$1.5\times {10}^{-4}$	$2.0\times {10}^{-3}$	$4.9\times {10}^{-4}$
	40	$7.7\times {10}^{-4}$	$1.7\times {10}^{-4}$	$2.5\times {10}^{-3}$	$6.4\times {10}^{-4}$
	50	$4.5\times {10}^{-4}$	$1.2\times {10}^{-4}$	$1.1\times {10}^{-3}$	$3.1\times {10}^{-4}$

,

Table 8. AO parameter analysis for case 7.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$1.9\times {10}^{-3}$	$1.7\times {10}^{-4}$	$5.3\times {10}^{-3}$	$1.5\times {10}^{-3}$
	40	$1.4\times {10}^{-3}$	$7.2\times {10}^{-5}$	$3.9\times {10}^{-3}$	$1.2\times {10}^{-3}$
	50	$6.9\times {10}^{-4}$	$2.3\times {10}^{-5}$	$1.8\times {10}^{-3}$	$4.9\times {10}^{-4}$
1500	30	$1.0\times {10}^{-3}$	$6.7\times {10}^{-5}$	$5.2\times {10}^{-3}$	$1.3\times {10}^{-3}$
	40	$8.5\times {10}^{-4}$	$7.4\times {10}^{-6}$	$2.5\times {10}^{-3}$	$6.7\times {10}^{-4}$
	50	$6.4\times {10}^{-4}$	$1.3\times {10}^{-4}$	$1.8\times {10}^{-3}$	$5.2\times {10}^{-4}$
2000	30	$8.5\times {10}^{-4}$	$3.9\times {10}^{-5}$	$2.2\times {10}^{-3}$	$5.6\times {10}^{-4}$
	40	$5.2\times {10}^{-4}$	$3.1\times {10}^{-5}$	$1.0\times {10}^{-3}$	$3.3\times {10}^{-4}$
	50	$5.6\times {10}^{-4}$	$2.1\times {10}^{-5}$	$2.4\times {10}^{-3}$	$5.7\times {10}^{-4}$

,

Table 9. AO parameter analysis for case 8.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$7.3\times {10}^{-4}$	$2.6\times {10}^{-5}$	$3.0\times {10}^{-3}$	$9.0\times {10}^{-4}$
	40	$9.1\times {10}^{-4}$	$2.4\times {10}^{-4}$	$2.5\times {10}^{-3}$	$7.4\times {10}^{-4}$
	50	$6.9\times {10}^{-4}$	$4.3\times {10}^{-5}$	$1.7\times {10}^{-3}$	$5.3\times {10}^{-4}$
1500	30	$5.5\times {10}^{-4}$	$3.0\times {10}^{-5}$	$1.8\times {10}^{-3}$	$4.7\times {10}^{-4}$
	40	$5.3\times {10}^{-4}$	$1.0\times {10}^{-4}$	$1.1\times {10}^{-3}$	$2.5\times {10}^{-4}$
	50	$4.8\times {10}^{-4}$	$7.8\times {10}^{-5}$	$1.6\times {10}^{-3}$	$4.6\times {10}^{-4}$
2000	30	$4.2\times {10}^{-4}$	$8.9\times {10}^{-5}$	$1.5\times {10}^{-3}$	$3.5\times {10}^{-4}$
	40	$5.1\times {10}^{-4}$	$6.8\times {10}^{-5}$	$2.3\times {10}^{-3}$	$5.5\times {10}^{-4}$
	50	$3.7\times {10}^{-4}$	$4.8\times {10}^{-5}$	$8.6\times {10}^{-4}$	$2.1\times {10}^{-4}$

and

Table 10. AO parameter analysis for case 9.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$6.9\times {10}^{-4}$	$1.1\times {10}^{-4}$	$2.6\times {10}^{-3}$	$6.2\times {10}^{-4}$
	40	$4.7\times {10}^{-4}$	$4.3\times {10}^{-5}$	$1.5\times {10}^{-3}$	$3.8\times {10}^{-4}$
	50	$5.2\times {10}^{-4}$	$3.5\times {10}^{-5}$	$2.2\times {10}^{-3}$	$5.7\times {10}^{-4}$
1500	30	$6.7\times {10}^{-4}$	$1.4\times {10}^{-4}$	$2.3\times {10}^{-3}$	$5.7\times {10}^{-4}$
	40	$4.6\times {10}^{-4}$	$2.8\times {10}^{-5}$	$1.4\times {10}^{-3}$	$4.3\times {10}^{-4}$
	50	$3.0\times {10}^{-4}$	$7.7\times {10}^{-5}$	$7.1\times {10}^{-3}$	$2.1\times {10}^{-4}$
2000	30	$3.3\times {10}^{-4}$	$3.6\times {10}^{-5}$	$9.6\times {10}^{-4}$	$2.3\times {10}^{-4}$
	40	$1.9\times {10}^{-4}$	$3.9\times {10}^{-5}$	$5.0\times {10}^{-4}$	$1.4\times {10}^{-4}$
	50	$2.8\times {10}^{-4}$	$4.5\times {10}^{-5}$	$1.1\times {10}^{-3}$	$2.5\times {10}^{-4}$

. Varying $\mathsf{\beta }$ and $\mathsf{\mu }$ causes the fit to vary with regard to a change in a generation or a population size, and the tables show the fitness trends. It is observed from the results given in Table 2, Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9 and Table 10 that the optimal fit for different generations and populations is achieved with case 9, i.e., $\mathsf{\beta }=0.1$ and $\mathsf{\mu }=0.1$.

Apart from the fitness results presented in Table 2, Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9 and Table 10, the mean fit values achieved with nine ($\mathsf{\beta }$ and $\mathsf{\mu }$) variations, three generations, and three populations are presented in

Table 11. AO parameter tuning mean value analysis.

Cases	Mean Fitness Value
Case 1	$908\times {10}^{-6}$
Case 2	$2.06\times {10}^{-3}$
Case 3	$938\times {10}^{-6}$
Case 4	$2.06\times {10}^{-3}$
Case 5	$479\times {10}^{-6}$
Case 6	$908\times {10}^{-6}$
Case 7	$938\times {10}^{-6}$
Case 8	$581\times {10}^{-6}$
Case 9	$440\times {10}^{-6}$

. It is observed from the mean fit values in Table 11 that AO obtained the minimum mean fit for case 9, i.e., $440\times {10}^{-6}$. Therefore, for optimal AO performance, the remaining simulation results are presented with the best hyper-parameter values, i.e., $\mathsf{\beta }=0.1$ and $\mathsf{\mu }=0.1$.

The fitness plots for case 9 with zero noise for three generations and populations are shown in Figure 3 and Figure 4. The fitness curves with fixed generation size and varying population size are given in Figure 3a–c, and Figure 4a–c include fitness curves for the fixed population size and changing generation size. Figure 3 and Figure 4 show that with the best parameter setting of $\mathsf{\beta }=0.1$ and $\mathsf{\mu }=0.1$, the AO strategy achieves minimum fit for a large number of generations and populations.

4.2. Statistical Convergence Analysis

In this section, the performance of AO algorithm is assessed by introducing three noise levels to the CAR model. Moreover, the fit of AO is estimated through three variations of generation [1000, 1500, 2000] and population [30, 40, 50]. The evaluation metrics used to assess the performance of AO for CAR are average fit, best fit, worst fit, and standard deviation (STD).

The performance in terms of fit variations and standard deviations for the three noise levels, i.e., 0.04, 0.06, and 0.08, is demonstrated in

Table 12. AO analysis with respect to generation and population sizes at 0.04 noise variance.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$3.8\times {10}^{-3}$	$1.0\times {10}^{-3}$	$9.3\times {10}^{-3}$	$2.7\times {10}^{-3}$
	40	$2.0\times {10}^{-3}$	$1.1\times {10}^{-3}$	$3.7\times {10}^{-3}$	$7.4\times {10}^{-4}$
	50	$1.8\times {10}^{-3}$	$1.1\times {10}^{-3}$	$4.0\times {10}^{-3}$	$8.9\times {10}^{-4}$
1500	30	$2.0\times {10}^{-3}$	$1.1\times {10}^{-3}$	$5.3\times {10}^{-3}$	$1.1\times {10}^{-3}$
	40	$1.7\times {10}^{-3}$	$1.1\times {10}^{-3}$	$3.0\times {10}^{-3}$	$6.0\times {10}^{-4}$
	50	$1.7\times {10}^{-3}$	$1.2\times {10}^{-3}$	$2.4\times {10}^{-3}$	$4.5\times {10}^{-4}$
2000	30	$1.9\times {10}^{-3}$	$1.1\times {10}^{-3}$	$3.2\times {10}^{-3}$	$4.8\times {10}^{-4}$
	40	$1.7\times {10}^{-3}$	$1.0\times {10}^{-3}$	$3.7\times {10}^{-3}$	$8.4\times {10}^{-4}$
	50	$1.9\times {10}^{-3}$	$1.1\times {10}^{-3}$	$5.6\times {10}^{-3}$	$1.3\times {10}^{-3}$

,

Table 13. AO analysis with respect to generation and population sizes at 0.06 noise variance.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$4.6\times {10}^{-3}$	$2.3\times {10}^{-3}$	$10\times {10}^{-3}$	$2.4\times {10}^{-3}$
	40	$3.3\times {10}^{-3}$	$2.3\times {10}^{-3}$	$4.4\times {10}^{-3}$	$6.9\times {10}^{-4}$
	50	$3.3\times {10}^{-3}$	$2.3\times {10}^{-3}$	$5.9\times {10}^{-3}$	$1.0\times {10}^{-3}$
1500	30	$3.6\times {10}^{-3}$	$2.4\times {10}^{-3}$	$6.5\times {10}^{-3}$	$1.3\times {10}^{-3}$
	40	$3.4\times {10}^{-3}$	$2.5\times {10}^{-3}$	$8.1\times {10}^{-3}$	$1.4\times {10}^{-3}$
	50	$2.9\times {10}^{-3}$	$2.3\times {10}^{-3}$	$4.1\times {10}^{-3}$	$6.0\times {10}^{-4}$
2000	30	$3.1\times {10}^{-3}$	$2.3\times {10}^{-3}$	$7.1\times {10}^{-3}$	$1.2\times {10}^{-3}$
	40	$2.8\times {10}^{-3}$	$2.3\times {10}^{-3}$	$3.8\times {10}^{-3}$	$4.6\times {10}^{-4}$
	50	$2.7\times {10}^{-3}$	$2.3\times {10}^{-3}$	$3.2\times {10}^{-3}$	$2.7\times {10}^{-4}$

and

Table 14. AO analysis with respect to generation and population sizes at 0.08 noise variance.

Generations (T)	Population (Np)	Average Fitness	Best Fitness	Worst Fitness	STD
1000	30	$6.4\times {10}^{-3}$	$4.1\times {10}^{-3}$	$13.1\times {10}^{-3}$	$2.5\times {10}^{-3}$
	40	$5.0\times {10}^{-3}$	$4.1\times {10}^{-3}$	$7.0\times {10}^{-3}$	$7.8\times {10}^{-4}$
	50	$5.0\times {10}^{-3}$	$4.1\times {10}^{-3}$	$7.1\times {10}^{-3}$	$8.6\times {10}^{-4}$
1500	30	$5.1\times {10}^{-3}$	$4.2\times {10}^{-3}$	$8.2\times {10}^{-3}$	$1.0\times {10}^{-3}$
	40	$4.7\times {10}^{-3}$	$4.1\times {10}^{-3}$	$6.3\times {10}^{-3}$	$6.4\times {10}^{-4}$
	50	$4.8\times {10}^{-3}$	$4.0\times {10}^{-3}$	$6.8\times {10}^{-3}$	$7.1\times {10}^{-4}$
2000	30	$4.6\times {10}^{-3}$	$4.1\times {10}^{-3}$	$4.9\times {10}^{-3}$	$2.2\times {10}^{-4}$
	40	$4.6\times {10}^{-3}$	$4.1\times {10}^{-3}$	$5.6\times {10}^{-3}$	$3.9\times {10}^{-4}$
	50	$4.5\times {10}^{-3}$	$4.1\times {10}^{-3}$	$5.0\times {10}^{-3}$	$2.5\times {10}^{-4}$

, respectively. It is witnessed from Table 12, Table 13 and Table 14 that the AO fit decreases by increasing population and generation size. It is noticed from Table 12 that the minimum average fit, best fit, and worst fit achieved for noise level = 0.04 are $1.7\times {10}^{-3}$, $1.0\times {10}^{-3}$, and $3.0\times {10}^{-3}$, respectively. Similarly, the three best fit values for noise levels 0.06 and 0.08, given in Table 13 and Table 14, are $2.7\times {10}^{-3},2.3\times {10}^{-3},\mathrm{and} 3.2\times {10}^{-3}$ and $4.5\times {10}^{-3},4.1\times {10}^{-3},\mathrm{and} 4.9\times {10}^{-3}$, respectively.

The performance of the AO method in terms of best fit for three noise levels, i.e., 0.04, 0.06, and 0.08, is evaluated for the three variations in a generation, 1000, 1500, and 2000, and population size, 30, 40, and 50. Figure 5 shows the fitness plots. The fitness curves in Figure 5a–c represent the best fit of the AO algorithm for noise variance = 0.04. In contrast, Figure 5d–e show the best fit curves for noise variance = 0.06. Likewise, the best fit plots for noise variance = 0.08 are given in Figure 5g–i. Figure 5a–i shows that the AO fit for the three noise levels, i.e., 0.04, 0.06, and 0.08, decreases significantly with increasing generation and population size as well. However, better fit results are achieved for smaller values of noise with a greater number of generations and populations.

To confirm the natural behaviour of the AO strategy for different noise values, the performance of the AO method is also verified by fixing the population size (30, 40, or 50) and changing the generation size (1000, 1500, or 2000) for three values of noise variance (0.04, 0.06, and 0.08), and the fitness-based learning curves are presented in Figure 6. Figure 6a–c represent the AO fit with population = 30, and the fitness plots for population = 40 are given in Figure 6d,e. Figure 6g–i denotes the fitness plots for population = 50. It is realized from the fitness curves given in Figure 6a–i that for a fixed population and generation size, the AO fit for low levels of noise, i.e., 0.04 and 0.06, is quite lower than the fit for high noise, i.e., 0.08. Nevertheless, AO accomplishes the minimum fit for the smallest noise level, i.e., 0.04, for fixed population size. Therefore, it is confirmed from the curves in Figure 6 that the performance of AO degrades noticeably for higher noise values.

4.3. Results Comparison with other Heuristics

To further investigate the exploration and exploitation phase of the AO, it is compared with the arithmetic optimization algorithm (AOA) [44], the sine cosine algorithm (SCA) [45], and the reptile search algorithm (RSA) [46] for 15 independent runs with 3 variations of generation (1000, 1500, 2000) and population (30, 40, 50).

Table 15. Comparison of AO with AOA, SCA, and RSA against the true values for the CAR model at 0.04 noise variance.

Algorithm	Generations (T)	Population (Np)	Design Parameters				Best Fit
			r1	r2	s1	s2
AO	1000	30	1.362	0.767	1.692	2.338	$1.0\times {10}^{-3}$
		40	1.357	0.765	1.706	2.308	$1.1\times {10}^{-3}$
		50	1.353	0.764	1.712	2.298	$1.1\times {10}^{-3}$
	1500	30	1.361	0.759	1.702	2.319	$1.1\times {10}^{-3}$
		40	1.356	0.768	1.704	2.316	$1.1\times {10}^{-3}$
		50	1.357	0.759	1.703	2.302	$1.2\times {10}^{-3}$
	2000	30	1.366	0.771	1.723	2.321	$1.1\times {10}^{-3}$
		40	1.359	0.760	1.686	2.329	$1.0\times {10}^{-3}$
		50	1.348	0.757	1.717	2.284	$1.1\times {10}^{-3}$
RSA	1000	30	1.190	0.599	1.740	1.855	$5.3\times {10}^{-2}$
		40	1.376	0.769	1.619	2.418	$3.1\times {10}^{-3}$
		50	1.201	0.569	1.371	2.042	$1.0\times {10}^{-1}$
	1500	30	1.281	0.734	1.663	2.178	$1.9\times {10}^{-2}$
		40	1.294	0.649	1.384	2.381	$2.5\times {10}^{-2}$
		50	1.127	0.579	1.704	1.767	$8.3\times {10}^{-2}$
	2000	30	1.272	0.704	1.174	2.648	$5.4\times {10}^{-2}$
		40	1.381	0.769	1.526	2.510	$7.5\times {10}^{-3}$
		50	1.227	0.708	1.719	2.051	$3.3\times {10}^{-2}$
SCA	1000	30	1.341	0.715	1.646	2.278	$2.9\times {10}^{-3}$
		40	1.354	0.774	1.725	2.332	$3.4\times {10}^{-3}$
		50	1.394	0.766	1.678	2.397	$2.6\times {10}^{-3}$
	1500	30	1.376	0.759	1.662	2.443	$4.7\times {10}^{-3}$
		40	1.364	0.779	1.572	2.497	$2.7\times {10}^{-3}$
		50	1.347	0.742	1.626	2.355	$9.5\times {10}^{-4}$
	2000	30	1.350	0.753	1.694	2.334	$2.5\times {10}^{-3}$
		40	1.326	0.782	1.600	2.389	$4.1\times {10}^{-3}$
		50	1.312	0.704	1.691	2.205	$2.8\times {10}^{-3}$
AOA	1000	30	1.346	0.866	1.494	2.613	$1.9\times {10}^{-2}$
		40	1.449	0.881	1.465	2.833	$1.7\times {10}^{-2}$
		50	1.353	0.769	1.555	2.519	$8.3\times {10}^{-3}$
	1500	30	1.394	0.762	1.791	2.250	$6.6\times {10}^{-3}$
		40	1.345	0.726	1.649	2.358	$6.2\times {10}^{-3}$
		50	1.385	0.800	1.613	2.512	$3.5\times {10}^{-3}$
	2000	30	1.331	0.677	1.455	2.409	$1.5\times {10}^{-2}$
		40	1.383	0.724	1.569	2.433	$7.8\times {10}^{-3}$
		50	1.346	0.742	1.763	2.166	$8.6\times {10}^{-3}$
True Values			1.350	0.750	1.680	2.320	0

,

Table 16. Comparison of AO with AOA, SCA, and RSA against the true values for the CAR model at 0.06 noise variance.

Algorithm	Generations (T)	Population (Np)	Design Parameters				Best Fit
			r1	r2	s1	s2
AO	1000	30	1.371	0.771	1.701	2.339	$2.3\times {10}^{-3}$
		40	1.369	0.777	1.715	2.336	$2.3\times {10}^{-3}$
		50	1.368	0.773	1.700	2.340	$2.3\times {10}^{-3}$
	1500	30	1.366	0.771	1.744	2.300	$2.4\times {10}^{-3}$
		40	1.362	0.761	1.733	2.291	$2.5\times {10}^{-3}$
		50	1.356	0.769	1.718	2.305	$2.3\times {10}^{-3}$
	2000	30	1.367	0.770	1.711	2.327	$2.3\times {10}^{-3}$
		40	1.364	0.776	1.723	2.317	$2.3\times {10}^{-3}$
		50	1.363	0.772	1.706	2.333	$2.3\times {10}^{-3}$
RSA	1000	30	1.200	0.657	1.159	2.531	$8.1\times {10}^{-2}$
		40	1.202	0.600	1.789	1.882	$5.9\times {10}^{-2}$
		50	1.183	0.580	1.425	2.118	$6.1\times {10}^{-2}$
	1500	30	1.120	0.582	1.584	1.850	$9.7\times {10}^{-2}$
		40	1.253	0.645	1.091	2.681	$7.2\times {10}^{-2}$
		50	1.265	0.683	1.431	2.396	$2.4\times {10}^{-2}$
	2000	30	1.102	0.600	1.888	1.557	$1.3\times {10}^{-1}$
		40	1.353	0.762	1.341	2.726	$3.0\times {10}^{-2}$
		50	1.342	0.758	1.349	2.610	$2.7\times {10}^{-2}$
SCA	1000	30	1.341	0.715	1.646	2.278	$6.1\times {10}^{-3}$
		40	1.354	0.774	1.725	2.332	$4.5\times {10}^{-3}$
		50	1.394	0.766	1.678	2.397	$3.3\times {10}^{-3}$
	1500	30	1.376	0.759	1.662	2.443	$3.2\times {10}^{-3}$
		40	1.364	0.779	1.572	2.497	$2.3\times {10}^{-3}$
		50	1.347	0.742	1.626	2.355	$2.8\times {10}^{-3}$
	2000	30	1.350	0.753	1.694	2.334	$2.5\times {10}^{-3}$
		40	1.326	0.782	1.600	2.389	$2.1\times {10}^{-3}$
		50	1.312	0.704	1.691	2.205	$2.3\times {10}^{-3}$
AOA	1000	30	1.427	0.870	1.470	2.808	$1.0\times {10}^{-2}$
		40	1.430	0.811	1.295	2.854	$2.3\times {10}^{-2}$
		50	1.315	0.737	1.738	2.167	$6.0\times {10}^{-3}$
	1500	30	1.419	0.789	1.446	2.662	$1.2\times {10}^{-2}$
		40	1.339	0.779	1.702	2.328	$4.5\times {10}^{-3}$
		50	1.369	0.794	1.077	3.000	$1.1\times {10}^{-2}$
	2000	30	1.281	0.789	1.599	2.357	$1.4\times {10}^{-2}$
		40	1.351	0.684	1.579	2.307	$1.2\times {10}^{-2}$
		50	1.399	0.806	1.754	2.346	$7.6\times {10}^{-3}$
True Values			1.350	0.750	1.680	2.320	0

and

Table 17. Comparison of AO with AOA, SCA, and RSA against the true values for the CAR model at 0.08 noise variance.

Algorithm	Generations (T)	Population (Np)	Design Parameters				Best Fit
			r1	r2	s1	s2
AO	1000	30	1.357	0.772	1.754	2.271	$4.1\times {10}^{-3}$
		40	1.370	0.772	1.715	2.322	$4.1\times {10}^{-3}$
		50	1.364	0.771	1.712	2.318	$4.1\times {10}^{-3}$
	1500	30	1.380	0.784	1.737	2.337	$4.2\times {10}^{-3}$
		40	1.371	0.778	1.707	2.344	$4.1\times {10}^{-3}$
		50	1.372	0.776	1.722	2.328	$4.0\times {10}^{-3}$
	2000	30	1.367	0.776	1.741	2.312	$4.1\times {10}^{-3}$
		40	1.372	0.775	1.746	2.304	$4.1\times {10}^{-3}$
		50	1.358	0.776	1.763	2.272	$4.1\times {10}^{-3}$
RSA	1000	30	1.143	0.529	1.575	1.928	$1.0\times {10}^{-1}$
		40	1.148	0.604	2.135	1.461	$1.4\times {10}^{-1}$
		50	1.141	0.562	1.283	2.256	$1.0\times {10}^{-1}$
	1500	30	1.120	0.598	1.752	1.709	$1.1\times {10}^{-1}$
		40	1.277	0.690	1.318	2.498	$3.6\times {10}^{-2}$
		50	1.293	0.694	1.210	2.656	$4.7\times {10}^{-2}$
	2000	30	1.205	0.661	1.549	2.053	$7.1\times {10}^{-2}$
		40	1.323	0.706	1.275	2.556	$4.2\times {10}^{-2}$
		50	1.304	0.733	1.536	2.333	$2.3\times {10}^{-2}$
SCA	1000	30	1.416	0.831	1.623	2.574	$7.9\times {10}^{-3}$
		40	1.359	0.806	1.688	2.380	$6.4\times {10}^{-3}$
		50	1.350	0.738	1.705	2.289	$4.1\times {10}^{-3}$
	1500	30	1.364	0.801	1.602	2.486	$4.7\times {10}^{-3}$
		40	1.375	0.788	1.679	2.379	$4.0\times {10}^{-3}$
		50	1.317	0.755	1.695	2.255	$5.3\times {10}^{-3}$
	2000	30	1.344	0.773	1.623	2.366	$3.9\times {10}^{-3}$
		40	1.399	0.778	1.658	2.429	$4.5\times {10}^{-3}$
		50	1.340	0.713	1.667	2.250	$5.1\times {10}^{-3}$
AOA	1000	30	1.427	0.870	1.470	2.808	$2.1\times {10}^{-2}$
		40	1.430	0.811	1.295	2.854	$1.0\times {10}^{-2}$
		50	1.315	0.737	1.738	2.167	$1.1\times {10}^{-2}$
	1500	30	1.419	0.789	1.446	2.662	$1.4\times {10}^{-2}$
		40	1.339	0.779	1.702	2.328	$1.3\times {10}^{-2}$
		50	1.369	0.794	1.077	3.000	$1.3\times {10}^{-2}$
	2000	30	1.281	0.789	1.599	2.357	$7.6\times {10}^{-3}$
		40	1.351	0.684	1.579	2.307	$5.7\times {10}^{-3}$
		50	1.399	0.806	1.754	2.346	$9.8\times {10}^{-3}$
True Values			1.350	0.750	1.680	2.320	0

shows the performance of all algorithms in terms of estimated weights and best fit for 0.04, 0.06, and 0.08 noise levels. It is seen that the algorithm gives better results at low noise, i.e., 0.04, than at high noise. Moreover, for low noise, the estimated weights are closer to the true values with minimum fit.

Table 18. Comparison of AO with RSA, SCA, and AOA against average fit for the CAR model at 0.04 noise variance.

Generations (T)	Population (Np)	AO	RSA	AOA	SCA
1000	30	$3.8\times {10}^{-3}$	$6.1\times {10}^{-1}$	$4.1\times {10}^{-2}$	$1.0\times {10}^{-1}$
	40	$2.0\times {10}^{-3}$	$2.5\times {10}^{-1}$	$4.7\times {10}^{-2}$	$6.4\times {10}^{-2}$
	50	$1.8\times {10}^{-3}$	$2.7\times {10}^{-1}$	$2.8\times {10}^{-2}$	$1.0\times {10}^{-1}$
1500	30	$2.0\times {10}^{-3}$	$5.0\times {10}^{-1}$	$3.0\times {10}^{-2}$	$1.2\times {10}^{-1}$
	40	$1.7\times {10}^{-3}$	$2.9\times {10}^{-1}$	$3.3\times {10}^{-2}$	$5.9\times {10}^{-2}$
	50	$1.7\times {10}^{-3}$	$2.5\times {10}^{-1}$	$3.0\times {10}^{-2}$	$3.7\times {10}^{-2}$
2000	30	$1.9\times {10}^{-3}$	$3.7\times {10}^{-1}$	$2.9\times {10}^{-2}$	$1.2\times {10}^{-1}$
	40	$1.7\times {10}^{-3}$	$2.8\times {10}^{-1}$	$2.4\times {10}^{-2}$	$8.7\times {10}^{-2}$
	50	$1.9\times {10}^{-3}$	$2.1\times {10}^{-1}$	$2.2\times {10}^{-2}$	$6.5\times {10}^{-2}$

,

Table 19. Comparison of AO with RSA, SCA, and AOA against average fit for the CAR model at 0.06 noise variance.

Generations (T)	Population (Np)	AO	RSA	AOA	SCA
1000	30	$4.6\times {10}^{-3}$	$3.9\times {10}^{-1}$	$5.0\times {10}^{-2}$	$6.9\times {10}^{-2}$
	40	$3.3\times {10}^{-3}$	$2.5\times {10}^{-1}$	$5.1\times {10}^{-2}$	$1.0\times {10}^{-1}$
	50	$3.3\times {10}^{-3}$	$2.7\times {10}^{-1}$	$3.2\times {10}^{-2}$	$6.5\times {10}^{-2}$
1500	30	$3.6\times {10}^{-3}$	$6.5\times {10}^{-1}$	$3.6\times {10}^{-2}$	$1.7\times {10}^{-1}$
	40	$3.4\times {10}^{-3}$	$2.6\times {10}^{-1}$	$3.5\times {10}^{-2}$	$1.9\times {10}^{-1}$
	50	$2.9\times {10}^{-3}$	$2.3\times {10}^{-1}$	$2.8\times {10}^{-2}$	$7.8\times {10}^{-2}$
2000	30	$3.1\times {10}^{-3}$	$3.4\times {10}^{-1}$	$4.0\times {10}^{-2}$	$1.6\times {10}^{-1}$
	40	$2.8\times {10}^{-3}$	$5.4\times {10}^{-1}$	$3.5\times {10}^{-2}$	$7.0\times {10}^{-2}$
	50	$2.7\times {10}^{-3}$	$2.1\times {10}^{-1}$	$2.3\times {10}^{-2}$	$7.1\times {10}^{-3}$

and

Table 20. Comparison of AO with RSA, SCA, and AOA against average fit for the CAR model at 0.08 noise variance.

Generations (T)	Population (Np)	AO	RSA	AOA	SCA
1000	30	$6.4\times {10}^{-3}$	$4.3\times {10}^{-1}$	$4.3\times {10}^{-2}$	$2.3\times {10}^{-1}$
	40	$5.0\times {10}^{-3}$	$6.7\times {10}^{-1}$	$2.5\times {10}^{-2}$	$1.0\times {10}^{-1}$
	50	$5.0\times {10}^{-3}$	$4.3\times {10}^{-1}$	$3.1\times {10}^{-2}$	$1.0\times {10}^{-1}$
1500	30	$5.1\times {10}^{-3}$	$3.0\times {10}^{-1}$	$3.5\times {10}^{-2}$	$9.2\times {10}^{-2}$
	40	$4.7\times {10}^{-3}$	$2.5\times {10}^{-1}$	$3.1\times {10}^{-2}$	$6.5\times {10}^{-2}$
	50	$4.8\times {10}^{-3}$	$2.9\times {10}^{-1}$	$3.9\times {10}^{-2}$	$1.1\times {10}^{-1}$
2000	30	$4.6\times {10}^{-3}$	$4.3\times {10}^{-1}$	$2.9\times {10}^{-2}$	$1.2\times {10}^{-1}$
	40	$4.6\times {10}^{-3}$	$2.7\times {10}^{-1}$	$2.6\times {10}^{-2}$	$1.5\times {10}^{-1}$
	50	$4.5\times {10}^{-3}$	$2.6\times {10}^{-1}$	$2.5\times {10}^{-2}$	$1.3\times {10}^{-1}$

show the performance of the AO, AOA, RSA, and SCA algorithms in terms of average fit for all noise variances. It is seen that for all noise variances, the AO algorithm gives better results as compared with RSA, SCA, and AOA for generations and populations.

The statistical analysis of AO, SCA, AOA, and RSA for multiple runs, noise variances, and population sizes and constant generation size are shown in Figure 7. It is witnessed that for all noise variances, the AO achieves worse fit as compared with RSA, AOA, and SCA. It is also observed that by increasing the noise level degrades the performance of all algorithms. However, AO achieves optimal fit in all scenarios.

To further investigate the performance of AO vs. RSA, AO vs. AOA, and AO vs. SCA, a nonparametric Mann-Whitney U test [47] is performed on average fit values for all algorithms on noise variances 0.04, 0.06, and 0.08 with generations 1000, 1500, and 2000 and populations 30, 40, or 50. The Mann-Whitney U test is a parametric equivalent of two sample t test. The significance level is 0.01 and one tailed hypothesis is used. The computed z-score is −6.29719 and p-value is less than 0.00001. Moreover, the result is significant at p < 0.01 as presented in Figure 8, Figure 9 and Figure 10.

The results of detailed simulations and the statistics indicate that the AO based swarming optimization heuristics effectively estimates the parameters of the CAR systems. However, the real time implementation of the swarm optimization algorithms for practical system identification problems require further investigation.

5. Conclusions

Following are the conclusions drawn from the extensive simulation results presented in the last section:

The strength of swarm intelligence of the Aquila optimizer, AO, is effectively exploited for parameter estimation of control autoregressive, CAR, systems. Performance of the AO is enhanced by escalating the population and generation at the expense of computational cost. While, the optimal fitness for different generations and populations is achieved for $\mathsf{\beta }=0.1$ and $\mathsf{\mu }=0.1$ exploitation adjustable factors. The robustness and accuracy of the AO decreases by varying the noise level. The comparative study of the AO with other heuristics based on AOA, SCA and RSA established the efficacy of the proposed scheme. The statistical analysis through Mann-Whitney U test endorsed the reliability of the AO scheme for CAR system identification The current study expands the application domain of swarm intelligence based optimizers by exploiting the strength of AO approach for system identification. However, future work may consider applying the proposed methodology of the for solving complex problems [48,49,50,51,52,53,54].