Multi-Objective Optimization Using Cooperative Garden Balsam Optimization with Multiple Populations

Abstract

Traditional multi-objective evolutionary algorithms (MOEAs) consider multiple objectives as a whole when solving multi-objective optimization problems (MOPs). In this paper, the hybridization of garden balsam optimization (GBO) is presented to solve multi-objective optimization, applying multiple populations for multiple objectives individually. Moreover, in order to improve the diversity of the solutions, both crowding distance computations and epsilon dominance relations are adopted when updating the archive. Furthermore, an efficient selection procedure called co-evolutionary multi-swarm garden balsam optimization (CMGBO) is proposed to ensure the convergence of well-diversified Pareto regions. The performance of the used algorithm is validated on 12 test functions. The algorithm is employed to solve four real-world problems in engineering. The achieved consequences corroborate the advantage of the proposed algorithm with regard to convergence and diversity.

1. Introduction

In recent years, numerous real-life issues have been formulated as MOPs, involving different fields such as manufacturing process optimization [1,2]. In fact, in single-objective optimization problems, the only challenge is covering a single optimal solution. On the other hand, MOPs have multiple contradictory objectives which need to be optimized synchronously [3,4]. Therefore, two goals commonly known as convergence and diversity are set forth.

As it is well-known, several population-based algorithms have been developed for MOP solving, a popular one of which is the NSGAII [5]. In this method, the population is classified via the non-dominated sorting procedure through which a rank of non-dominance is attributed to each individual. Moreover, to assess the density of the region around a given individual, the ones in the same rank are classified based on the crowding distance operator. Moreover, to improve diversity, the SPEA II was proposed in [6], which includes new fitness assignment based on a density estimator, using a truncation technique when updating a predefined archive size. However, several attempts have been made towards the success of such approaches. For example, Pareto solutions were stored in an external repository in multiple objective PSO (MOPSO) [7]. An effective selection strategy of Gbest, in which the objective space is divided in adaptive grids, has also been employed to maintain the diversity of solutions. In addition, to improve both convergence and diversity, the ε-dominance multiple objective evolutionary algorithm (ε-MOEA) was developed, using the ε-dominance relation [8]. However, in another example, to decompose the master multi-objective problem into several weighted sub-problems and, subsequently, to solve them simultaneously, the MOEA/D framework was proposed based on decomposition [9].

In multi-objective optimal processing, the preference-inspired co-evolutionary algorithm using goal vectors (PICEA-g) [10] is utilized to address a problem. Based on NN-DNSGA-II [11], an artificial neural network is combined with the NSGA II algorithm. However, the Indicator-Based Evolutionary Algorithm (IBEA) [12] adopts a new approach, in which indicators are directly used in the optimization process. Moreover, to solve multi-objective scheduling problems, the Chaotic PICEA-g-based algorithm was introduced in [10].

In recent years, new multi-objective methods have emerged mainly based on the swarm intelligence algorithm. Similar to MOPSO, the multi-objective Grey Wolf Optimizer (MOGWO) was proposed by [13]. Moreover, the multi-objective AntLion Optimizer (MOALO) was set forth by [14], in which a reference frame and a solution-coverage-based roulette selection are used to store the Pareto solutions and to continue the circulation of solutions, respectively. Based on the chicken swarm optimizer, it was combined with the external population in [15], using crowding calculations and ε-dominance to continue diversity to solve MOPs. In this context, in order to achieve the engineering design problem, the whale optimization algorithm was used in [16]. Moreover, the sorting and updating method was adopted based on NSGA II crowding distance to guide the population to the region with good diversity in the search space. Other well-known proposals can also be referred to, among which are the multi-objective Spotted Hyena Optimizer (MOSHO) [17], the hybrid multi-objective firefly (HMOFA) [18], the hybrid multi-objective cuckoo search (HMOCS) [19], the multi-objective Salp Swarm Algorithm (MSSA) [20], the multi-objective Artificial Sheep Algorithm (MOASA) [21] and the multi-objective Seagull Optimization Algorithm (MOSOA) [22].

Most of the MOEAs mentioned above suffer from fitness allocation problems. In their application, the efficiency of the search engine is substantially low unless fitness allocation is properly designed. To avoid this issue, Zhan et al. proposed the co-evolutionary multiswarm PSO (CMPSO) algorithm [23], in which multiple populations are used for independent objectives to be simultaneously optimized, and an external archive is set to preserve the achieved non-dominant solutions. Moreover, the archived information is used to lead the independent evolution of each population and to share information among various populations. Inspired by Zhan et al., a multi-objective cooperative differential evolution (CMODE) method was proposed by Wang et al. [24] to solve MOPs. In the method, the search engine is replaced by the adaptive differential evolution algorithm, which is similar to CMPSO.

Based on the achieved consequences, and inspired by CMPSO and CMODE, a hybrid cooperative multi-objective GBO (CMGBO) is introduced, in which multi-population studies are conducted to meet multiple objectives. The selection of the GBO algorithm is mainly motivated by its novelty and robustness in converging to potentially optimal solutions as well as in discovering promising regions in the search space. The application of these methods is justified by their encouraging consequences achieved by the use of ε-dominance and the notion of archives in the field of multi-objective optimization. Hence, the main contributions of this study can be summarized as follows:

• Cooperative multi-objective garden balsam optimization (CMGBO) is proposed for solving MOPs;
• To maintain a limited size of the Pareto set, an external archive is integrated into the CMGBO algorithm;
• The ε-dominance relationship is adopted to update the archive;
• Crowding distance calculations are used within the objective space to improve the diversity of solutions;
• An effective strategy of the leader’s selection is proposed to further improve both the convergence and diversity of the Pareto front.

The rest of the paper is structured as follows. Section 2 introduces the fundamental concepts of MOPs and offers an overview of the GBO algorithm. Section 3 outlines the proposed CMGBO algorithm. The analyses and discussions of the experimental consequences are presented in Section 4. Finally, Section 5 concludes the paper.

2. Background

2.1. Multi-Objective Optimization Problem (MOP)

MOP consists of finding a set of decision vectors which optimize the objective vector, whose elements represent an ensemble of conflicting objective functions.

Mathematically, an MOP (for minimization case) is described as below [25]:

$$\begin{array}{l}\min F\left(x\right)={\left[f_1\left(x\right),f_2\left(x\right),\cdots ,f_m\left(x\right)\right]}^{\mathrm{T}}\\ \mathrm{s}.\mathrm{t}.g_j\left(x\right)\le 0,j=1,2\cdots ,p\end{array}$$

(1)

where $x={\left[x_1,x_2,\dots ,x_n\right]}^{\mathrm{T}}\in X$ is the decision vector made up of $n$ decision variables, $X$ represents the decision space, $f_i\left(x\right),i=1,2,3,\dots ,m$ signifies the objective function, m denotes the number of objective functions and $g_j\left(x\right)$ indicates the constraint function. Since several objective functions are simultaneously carried out by multi-objective optimization, the concept of dominance is used to compare the solutions.

In the minimization problem, a decision vector $u\in X$ dominates another vector $v\in X$ (symbolized by $u\prec v$), only if:

$$f_i\left(u\right)\le f_i\left(v\right),\forall i\in \left\{1,2,\cdots ,m\right\}$$

(2)

The solution that is not dominated by other feasible ones is called non-dominated or the Pareto solution. Furthermore, the set of non-dominated solutions PS is known as the Pareto set and is defined as:

$$PS=\left\{u\in X,\exists v\in X,u\prec v\right\}$$

(3)

The image of the Pareto set PS by the objective vector represents the Pareto front (commonly known as Pareto front PF).

2.2. Epsilon Dominance

The ε-dominance is a relaxation relation proposed in [26], through which the performance of multi-objective evolutionary algorithms (MOEAs) can be improved. For a given relaxation $\epsilon \in R^m$ vector (m being the number of objective functions, and ${\epsilon }_i>0$), the solution $u$ is said to $\epsilon$-dominate $v$ (denoted by $u{\prec }_{\epsilon }v$) if the condition $f_i\left(u\right)-\epsilon \le f_i\left(v\right)$ is verified for all objective functions $f_i$.

ε-dominance is an effective technique for maintaining the diversity of multi-objective optimization algorithms without losing convergence properties toward the Pareto-optimal set. Furthermore, due to its uncomplicated implementation, decision makers can intuitively control the number of available solutions.

2.3. Performance Metrics

In general, the performance of a multi-objective optimization method is evaluated by assessing its convergence, uniformity and spread. This method examines the approximation degree (distance) between the solution set and the real Pareto front, the uniformity of the solution set distribution on PF and the wide distribution of the solution set in the target space. Generally, the achieved solution set is expected to be as close to the PF, as evenly distributed on the PF and as complete to express the PF as possible. In our case, two widely used performance metrics, namely IGD and HV, are adopted for the comparison.

• Inverted Generational Distance (IGD)

The IGD metric is one of the most used metrics in the field of MOPs, through which measuring both the convergence and diversity of the algorithms is carried out. It is mathematically formulated as [27]:

$$\mathrm{IGD}\left(A,P^{*}\right)=\frac{{\sum }_{i=1}^{\left|P^{*}\right|}\min \left(\mathrm{Dist}\left(p_i\right)\right)}{\left|P^{*}\right|},\mathrm{Dist}\left(p_i\right)=\left\{d\left(p_i,a_j\right)|a_j\in A,j=1,2,\cdots ,\left|A\right|\right\}$$

(4)

where $A$ is the solution set achieved by the algorithm, $P^{*}$ denotes the solution set uniformly sampled along the real PF and $d\left(p_i,a_j\right)$ indicates the Euclidean distance between the two points. Note that a lower IGD stands for better performance.

• Hyper-Volume Metric (HV)

HV is used to calculate the volume of the region in the target space surrounded by the non-dominated solution set and the reference points. Although it can simultaneously evaluate convergence and diversity without providing real PF, it has high computational complexity [28]. Mathematically, HV is formulated as:

$$\mathrm{HV}=\delta \left({\displaystyle {\cup }_{i=1}^{\left|S\right|}{v}_i}\right)$$

(5)

where $\delta$ is the Lebesgue measure, s denotes a non-dominant solution set and $v_i$ denotes the super volume.

2.4. Garden Balsam Optimization

In the simulation of garden balsam expansion and propagation, the algorithm iterates from the beginning. In the process, a mechanical and a secondary propagator, a mapping rule and a selection strategy are adopted in turn until the termination condition is satisfied, i.e., either the accuracy requirement of the problem is satisfied or the maximum iteration is attained [29,30]. The flow chart of the GBO algorithm is shown in Figure 1.

Here are the steps involved in the dispersal of the garden balsam population:

(1)

The initialization of a population is due to the few seeds scattered randomly on a specific area growing roots and producing a first-generation population;

(2)

Progeny reproduction: The natural conditions in the growing area cause each plant in the first-generation population to show different growth rates. The stronger plants bear more fruits and spawn more seeds.

As a consequence of individual $x$, the following number of seeds are produced:

$$S=\frac{f_{max}-f\left(x\right)}{f_{max}-f_{min}}\times \left(S_{max}-S_{min}\right)+S_{min}$$

(6)

where $f\left(x\right)$ is the fitness value, $f_{max}$ signifies the current population’s maximum fitness value, $f_{min}$ indicates the current population’s minimum fitness value, $S_{max}$ means the upper limit on the number of seeds and $S_{min}$ stands for the minimum number of seeds.

(3)

Mechanical transmission: Plants in good growing conditions bear fully grown fruits, more powerful ejection forces and, consequently, farther ejections of seeds.

The range of seed diffusion is calculated as follows:

$$A={\left(\frac{ite{r}_{max}-iter}{ite{r}_{max}}\right)}^n\times \frac{f_{max}-f\left(x\right)}{f_{max}-f_{min}}\times A_{\mathrm{init}}$$

(7)

When $f_{max}-f\left(x\right)=0$, or $ite{r}_{max}-iter=0$, $A=\mathsf{\epsilon }$. $\epsilon$ denotes the min value, $iter$ denotes the evolutionary iterations at present, $ite{r}_{max}$ stands for the max iteration and $n$ suggests a nonlinear harmonic factor.

(4)

Second transmission: For population diversity to increase, seeds are randomly transported from place to place by animals, water and wind.

Its manifestation is as follows:

$$x_1^{\prime }=x_B+F\left(x_2-x_3\right)$$

(8)

where $x_1^{\prime }$ is the new position of $x_1$ after the second transmission, $x_B$ denotes the best position, $F$ stands for the zoom factor and $x_2$ and $x_3$ are the positions of two dissimilar seeds.

(5)

Competition-based elimination: The population size of a specific region is limited by $N_{max}$. When the population size reaches the upper limit, elite seeds are retained, and redundant seeds are randomly eliminated. The number of elite seeds is calculated using Formula (9).

$$N_{best}=\frac{iter}{ite{r}_{max}}{N}_{max}$$

(9)

where $N_{best}$ indicates the number of elite solutions, and $iter$ and $ite{r}_{max}$ are similar to those in Formula (7).

3. CMGBO: Cooperative Multi-Objective Garden Balsam Optimization

The details of the proposed ε-dominance-based CMGBO algorithm are outlined in this section. To handle multiple objectives, multiple populations are carried out in CMGBO. Moreover, in order to optimize the corresponding objective, a GBO is run for each population. In addition, an archive is also integrated to save and improve all those that are non-dominated. By using information from other populations through the archive, each population is searched along the PF. Figure 2 demonstrates the framework of CMGBO, in which Subpop and obj represent the population and optimization target. The stages through which the proposed cooperative multi-objective garden balsam optimization (CMGBO) operates are described in the following subsections.

3.1. Co-Evolutionary Mechanism

For the convenience of description, the evolution process of GBO is analyzed through giving an example of the mth population. Starting from initialization, the CMGBO randomly generates the mth population for the mth optimization objective within a given range. At the end of each evolution, its minimum and maximum values are determined. Subsequently, Formulas (6) and (7) are used to calculate $S_i$ and $A_{m,i}$.

At this moment, the seed dispersion range of the ith solution $X_{m,i}$ is as follows:

$$X_{m,i}\left(iter+1\right)=X_{m,i}\left(iter\right)+A_{m,i}\left(iter\right)\times U\left(-1,1\right)+P\left(A_{m,r}\left(iter\right)-X_{m,i}\left(iter\right)\right)$$

(10)

where P is the scale parameter, $A_{m,r}\left(iter\right)$ denotes any solution in the archive and $P\left(A_{m,r}\left(iter\right)-X_{m,i}\left(iter\right)\right)$ represents the shared information fed back by other sub-populations. Thus, each population dispersion can not only access and search information from its own history, but it is also able to share the search information of other sub-populations. Accordingly, each solution focuses on spreading new seeds throughout the PF and attempts to avoid being drawn to the edge only by the search information of the current group.

3.2. Updating the Archive

For the CMGBO algorithm, a limited-size external population called “archive” is used. Two objectives are served by this archive: it saves the non-dominant solutions as well as updates the seed population. This external archive needs to be updated in each iteration; therefore, the ε-dominance approach is used to update the archive’s solutions. For each solution in the archive, we associate an identification vector $O={\left[O_1,O_2,\dots ,O_m\right]}^T$ with $m$ as the number of objectives.

$$O_i\left(f\right)=\left[\frac{\log \left(f_i\right)}{\log \left(\epsilon +1\right)}\right]$$

(11)

In the proposed archiving strategy, each of the solutions is compared to all of the archive’s elements. The details of the proposed archiving strategy, including all the possible cases, are presented in Algorithm 1.

Algorithm 1. Refresh the archive population pseudo-code.

Require: A(t): in iteration t, the group of outside archive; $v$: the new seed.

1:  if $\exists a\in A\left(t\right),O_a\succ O_v$ then

2:    $v$ is not recognized

3:  else if $\exists a\in A\left(t\right),O_v\succ O_a$ then

4:    $v$ exchanges $a$

5:  else if $\exists a\in A\left(t\right),O_v\sim O_a$ then

6:    if $v\sim s$ then

7:       Keep the seed with the smallest distance from $O$

8:    else

9:       Keep the seed that dominates the other

10:   end if

11: else

12:   Add the seed $v$ in A(t)

13: end if

3.3. Crowding Distance Operator

To assess the solutions’ density around a given solution in the objective space, a crowding distance mechanism is used as the sorting operator of the archive. The crowding distance computation is presented in Algorithm 2.

Algorithm 2. Crowding distance computation algorithm.

Require: A: the external archive

1:  $I=‖A‖$

2:  for $a_i\in A$ do

3:      $A{\left[a_i\right]}_{distance}=0$

4:  end for

5:  for each objective m do

6:      $A=\mathrm{Sort}\left(A,m\right)$

7:      $A{\left[1\right]}_{distance}=A{\left[I\right]}_{distance}=\infty$

8:      for $i=2$ to $\left(I-1\right)$do

9:         $A{\left[i\right]}_{distance}=A{\left[i\right]}_{distance}+\left(A{\left[i+1\right]}_m-A{\left[i-1\right]}_m\right)$

10:     end for

11: end for

To improve the diversity of the method, the crowding distance is integrated into the updating archive procedure in our proposal. The crowding distance is first calculated for each solution in the archive. Subsequently, a sorting procedure is carried out to sort the archive based on the crowding distance. Next, the last CurrSize-MaxSize solutions are eliminated from the archive. Applying this strategy, the most crowded (CurrSize-MaxSize) solutions in the objective space are removed, leading to better diversity.

3.4. General Operating of the CMGBO Algorithm

The CMGBO algorithm starts with randomly generating the set of seeds. Grounded on epsilon dominance, the method identifies the non-dominated solutions to prepare the archive for the main loop. Subsequently, the new seed’s positions are calculated according to the GBO standard presented earlier in this paper. New seeds are considered as the archive candidates, applying the updated archive procedure presented earlier. Next, in each iteration, in order to sort the archive in decreasing order based on the computed crowding distance, the crowding distance of each solution is calculated. Therefore, in order to remove the most crowded seeds from the outsized archive, deletion is employed as the method. Finally, as shown in Algorithm 3, the external archive containing the consequences of the Pareto front is returned by the algorithm.

Algorithm 3. CMGBO algorithm.

Require: A; iter = 0; number of function evaluation, NFE = 0;

1:  for $m=1\to M$ do %M is the number of objectives%

2:   for $i=1\to N_{init}$ do %$N_{init}$ is the pop size%

3:    randomly initialize individual $po{p}_m\left(i\right)$ for the $m$th objective;

4:    compute objective values of $po{p}_m\left(i\right)$;

5:    NFE = NFE + 1;

6:   end for

7:  end for

8:  refresh A by sub-populations (${\displaystyle \underset{m=1}{\overset{M}{\cup }}po{p}_m}$);

9:  while iter < itermax do

10:  for $m=1\to M$ do

11:   $see{d}_m=\varphi$; % ϕ empty set%

12:   calculate the fitness value of each seeds;

13:   for$i=1\to \left|po{p}_m\right|$ do

14:    compute $S\left(x_i\right)$ of $x_i\in po{p}_m$;

15:    compute the dispersion range of new seeds, $A\left(x_i\right)$ of $x_i\in po{p}_m$;

16:    for do

17:     generate new seeds of $i$th parent in $m$th population;

18:    end for

19:    $NFE=NFE+n\left(x_i\right)$;

20:   end for

21:   for$j=1\to N_{sec}$ do

22:    randomly select a seed for second transmission;

23:   end for

24:   $NFE=NFE+N_{sec}$;

25:   evaluate the new offspring $see{d}_m$;

26:   $po{p}_m^{new}=\left(po{p}_m\cup see{d}_m\right)$;

27:   sort the whole population ($po{p}_m^{new}$);

28:   Calculate the number of elite solutions $N_{best}$;

29:   $N_{best}$ solutions with the optimal adaptive value from $po{p}_m^{new}$ are selected to form the elite solution set $E_m$;

30:   if $N_{max}<\left|po{p}_m^{new}\right|$ then

31:    truncate the population with elite-random selection algorithm until $\left|po{p}_m^{new}\right|=N_{max}$;

32:   end if

33:  end for

34:  update archive population according to Algorithm 1;

35:  $iter=iter+1$;

36: end while

Output: the members of A.

4. Consequences and Discussion

4.1. Experiment Settings

In order to examine the proposed algorithm, four algorithms, MOEA with decomposition and DE operators (MOEA/D-DE) [31], CMODE [24], multi-objective comprehensive learning PSO (MOCLPSO) [32], and CMPSO [23], were compared. As summarized in

Table 1. Preferences settings.

Algorithm	Parameters
MOEA/D-DE	$\begin{array}{l}N=100, CR=1.0, F=0.5, h=20,\\ p_m=\frac{1}{D}, T=20, \delta =0.9, \mathrm{and} n_r=2\end{array}$
CMODE	$\begin{array}{l}N=20, \left|A\right|=100, CR=1.0, F=0.5, h=20,\\ p_m=\frac{1}{D}, T=20, \delta =0.9, \mathrm{and} n_r=2\end{array}$
MOCLPSO	$\begin{array}{l}N=100, p_c=0.1, p_m=0.4, \omega =0.9\to 0.2,\\ \mathrm{and} c=2\end{array}$
CMPSO	$\begin{array}{l}N=20, \left|A\right|=100, \omega =0.9\to 0.2,\\ \mathrm{and} c_1=c_2=c_3=2.0\end{array}$
CMGBO	$\begin{array}{l}{N}_{init}=5, N_{max}=20, \left|A\right|=100, N_{sec}=5,\\ n=3, F=2, \mathrm{and} P=2\end{array}$

, the parameters of the mentioned algorithms were set. In addition, in order to score the performance of the method by using IGD and HV indicator metrics, ZDTs (two-objective test functions) [25], UFs (two-objective test functions) [33], DTLZs (tri-objective test functions) [34] and MaFs (multi-objective test functions) [35] were used as the benchmark test functions. In total, 15 test problems (5 ZDTs, 2 UFs, 2 DTLZs and 6 MaFs) were carried out, the characteristics of which are described in

Table 2. Characteristics of multi-objective test functions.

Test Function	Variables M	Variables D	Range	Optimum	Characteristics
ZDT1	2	30	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0,\\ 2\le i\le D\end{array}$	Convex
ZDT2	2	30	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0,\\ 2\le i\le D\end{array}$	Concave
ZDT3	2	10	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0,\\ 2\le i\le D\end{array}$	Convex, Disconnected
ZDT4	2	10	$\begin{array}{l}{x}_1\in \left[0,1\right],\\ x_i\in \left[-5,5\right],\\ 2\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0,\\ 2\le i\le D\end{array}$	Concave
ZDT6	2	10	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0,\\ 2\le i\le D\end{array}$	Concave
UF1 (DE)	2	30	$\begin{array}{l}{x}_1\in \left[0,1\right],\\ x_i\in \left[-1,1\right],\\ 2\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right],\\ x_i=\sin \left(6\pi x_1+\frac{i\pi }{D}\right),\\ 2\le i\le D\end{array}$	Convex, Disconnected
UF5 (DE)	2	30	$\begin{array}{l}{x}_1\in \left[0,1\right],\\ x_i\in \left[-1,1\right],\\ 2\le i\le D\end{array}$	$\begin{array}{l}\left(F_1,F_2\right)=\left(\frac{i}{2N},1-\frac{i}{2N}\right),\\ 0\le i\le 2N, N=10\end{array}$	Scatter, Disconnected
DTLZ1	3	12	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Linear
DTLZ7	3	12	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Concave, Disconnected
MaF1	10	19	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Linear
MaF2	10	19	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Concave
MaF3	10	19	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Convex, Multimodal
MaF4	10	19	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Concave, Multimodal, Badly Scaled
MaF5	10	19	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Convex, Biased, Badly Scaled
MaF6	10	19	$\begin{array}{l}{x}_i\in \left[0,1\right],\\ 1\le i\le D\end{array}$	$\begin{array}{l}{x}_1\in \left[0,1\right], x_i=0.5,\\ 2\le i\le D\end{array}$	Concave, Degenerate

. M is the number of objectives. Finally, all of the algorithms were executed 30 times.

4.2. Experimental Results on ZDT Problems

Table 3. Consequences on the ZDT problems.

Test Function		MOEA/D-DE	CMODE	MOCLPSO	CMPSO	CMGBO
ZDT1	Mean	1.60 × 10−1	4.21 × 10−3	4.80 × 10−3	4.13 × 10−3	4.07 × 10−3
	Std	1.93 × 10−2	2.54 × 10−4	1.76 × 10−4	8.30 × 10−5	2.91 × 10−4
	Rank	5−	3−	4−	$2\approx$	1
ZDT2	Mean	2.30 × 10−1	7.15 × 10−3	3.80 × 10−2	4.32 × 10−3	3.92 × 10−3
	Std	3.07 × 10−2	2.51 × 10−4	3.04 × 10−3	1.03 × 10−4	3.25 × 10−4
	Rank	5−	3−	4−	$2\approx$	1
ZDT3	Mean	2.30 × 10−1	9.12 × 10−2	5.49 × 10−3	1.39 × 10−2	2.25 × 10−2
	Std	2.17 × 10−2	5.36 × 10−2	2.49 × 10−4	3.49 × 10−3	4.47 × 10−3
	Rank	5−	4−	1+	2+	3
ZDT4	Mean	3.10 × 10−1	4.90 × 10−1	3.26	7.90 × 10−1	5.70 × 10−1
	Std	2.30 × 10−1	3.20 × 10−1	1.35	2.60 × 10−1	2.20 × 10−1
	Rank	1+	2+	5−	4−	3
ZDT6	Mean	1.54	4.06 × 10−3	3.69 × 10−3	3.72 × 10−3	3.64 × 10−3
	Std	1.30 × 10−1	5.19 × 10−4	1.31 × 10−4	1.47 × 10−4	1.73 × 10−4
	Rank	5−	4−	$2\approx$	$3\approx$	1
Final Rank	Total	21	16	16	13	9
	Final	5	3	3	2	1
Better-Worse		−3	−3	−2	0
Algorithm		MOEA/D-DE	CMODE	MOCLPSO	CMPSO	CMGBO

‘+’, ‘−’ and ‘$\approx$’ indicate that the values achieved from the method are distinctly superior to, inferior to and analogous to those of CMGBO by the Wilcoxon rank sum test with α = 0.05, respectively. The number of ‘+’ minus ‘−’ is shown by the ‘Better-Worse’ row. Values smaller than 0 mean that it is generally beaten by CMGBO.

demonstrates a comparison of the experimental consequences of the five methods employed to solve the ZDT problem. The consequences show the satisfactory application prospects of CMGBO in the ZDT of the convex and concave PFs, the best performance of which was on ZDT1 of the convex PF and ZDT2 and ZDT6 of the concave PF. Single-peak double targets were observed for the three problems, indicating the strong approximation ability of CMGBO to the PF of multi-objective problems with simple targets. In addition, CMGBO was the third best on ZDT3, whose PF was not connected convex, which was not much different from MOCLPSO, which displayed the best performance. As can be observed in Table 3, all the algorithms performed poorly on ZDT4, with CMGBO also being the third best. This may be due to the multi-modal Rastrigin functions of the local PFs of ZDT4. In addition, considering the overall and average performances in all aspects, CMGBO demonstrated the best performance among the five algorithms. Moreover, comparable to CMPSO, the consequences of the test verified the significantly superior performance of CMGBO in the ZDT problem set compared with the other three competitors.

Figure 3 indicates non-dominated solutions achieved by different algorithms in solving the five ZDT problems. Since some of the algorithms demonstrated similar performances to CMGBO on the same problem, only CMGBO was used as a representative in the drawing. As can be observed in Figure 3, the solution achieved by CMGBO not only properly approximates the whole PF, it indicates good distribution on the whole PF.

4.3. Experimental Results on UF and DTLZ Problems

As can be observed in

Table 4. Consequences on UF and DTLZ problems.

Test Function		MOEA/D-DE	CMODE	MOCLPSO	CMPSO	CMGBO
UF1	Mean	6.95 × 10−2	4.68 × 10−2	8.52 × 10−2	7.57 × 10−2	4.79 × 10−2
	Std	3.00 × 10−2	8.48 × 10−3	2.41 × 10−2	2.01 × 10−2	1.96 × 10−2
	Rank	3−	$1\approx$	5−	4−	2
UF5	Mean	4.60 × 10−1	2.42 × 10−1	3.80 × 10−1	2.52 × 10−1	2.08 × 10−1
	Std	1.06 × 10−1	9.58 × 10−2	9.25 × 10−2	4.04 × 10−2	3.24 × 10−2
	Rank	5−	2−	4−	3−	1
DTLZ1	Mean	9.47 × 10−2	5.67 × 10−2	2.07 × 10−3	2.75 × 10−3	3.15 × 10−3
	Std	3.25 × 10−2	2.21 × 10−2	3.49 × 10−4	4.36 × 10−4	5.27 × 10−4
	Rank	5−	4−	1+	2+	3
DTLZ7	Mean	4.79 × 10−2	6.61 × 10−3	4.36 × 10−2	2.42 × 10−3	3.73 × 10−3
	Std	9.51 × 10−3	8.92 × 10−3	9.12 × 10−3	6.05 × 10−3	5.92 × 10−3
	Rank	5−	3−	4−	1+	2
Final Rank	Total	18	10	14	10	8
	Final	5	2	4	2	1
Better-Worse		−4	−3	−2	0
Algorithm		MOEA/D-DE	CMODE	MOCLPSO	CMPSO	CMGBO

‘+’, ‘−’ and ‘$\approx$’ indicate that the values achieved from the method are distinctly superior to, inferior to and analogous to those of CMGBO by the Wilcoxon rank sum test with α = 0.05, respectively. The number of ‘+’ minus ‘−’ is shown by the ‘Better-Worse’ row. Values smaller than 0 mean that it is generally beaten by CMGBO.

, CMODE performed the best on UF1, and CMGBO performed second on UF1. The performances on the DTLZ problems are also compared in this section. The consequences reveal that, although MOCLPSO displayed the best performance on DTLZ1, CMGBO ranked third. Moreover, CMPSO and CMGBO demonstrated the best and the second-best performances on DTLZ7, respectively. Moreover, CMGBO and CMPSO ranked first and second in UF and DTLZ on average. Being significantly better than the other three competitors, these two algorithms were also confirmed by the Wilcoxon rank sum test to occupy the first place in performance.

Figure 4 indicates the dominant solutions achieved by different methods in solving UF1 problems. CMODE and CMGBO displayed the best uniformity in UF1, and MOCLPSO showed the poorest. The solutions achieved by comparison methods in solving UF5 problems are indicated in Figure 5. In Figure 5e, it can be seen that the solution of CMGBO was closer to the Pareto front of UF5. Moreover, in Figure 5a, MOEA/D-DE demonstrated the poorest performance on UF5, and the approximation distance between the achieved solutions and the PF of UF5 was the largest.

Figure 6 indicates that the approximation degree between the solutions achieved by MOCLPSO and the Pareto front was the best when solving the DTLZ1 problem. CMPSO and CMGBO were placed at the second and the third ranks, respectively. However, the IGD values of the three are not significantly different. MOEA/D-DE and CMODE showed poor performances. This can be mainly attributed to the non-satisfactory distribution uniformity of the solutions achieved by MOEA/D-DE and CMODE. Furthermore, the local concentration phenomenon was obvious, resulting in their IGD values to be relatively large.

Figure 7 indicates the solutions achieved by different algorithms when solving the DTLZ7 problem. As can be observed, the solution achieved by CMGBO described in this study was relatively close to the real PF of DTLZ7, ranking it at second place next to CMPSO. However, the distribution of solutions achieved by MOEA/D-DE and CMODE between the Pareto fronts was not satisfactory. The non-dominant solutions achieved by MOEA/D-DE were mainly concentrated in the two regions, and the ones achieved by CMODE were distributed along the edges.

4.4. Experimental Results on the MaF Test Functions

The results of the IGD metric on the MaF test functions are shown in

Table 5. Statistical results for IGD on MaF test functions.

Algorithm		Best	Worst	Median	Average	Std.
MaF1	CMGBO	1.08 × 10−2	1.72 × 10−2	1.29 × 10−2	1.32 × 10−2	1.20 × 10−3
	CMPSO	8.88 × 10−3	1.34 × 10−2	1.03 × 10−2	1.06 × 10−2	1.26 × 10−3
	CMODE	5.05 × 10−2	7.54 × 10−2	6.48 × 10−2	6.42 × 10−2	5.87 × 10−3
	MOCLPSO	3.32 × 10−2	5.62 × 10−2	4.48 × 10−2	4.52 × 10−2	5.49 × 10−3
	MOEA/D-DE	2.28 × 10−2	3.48 × 10−2	2.65 × 10−2	2.76 × 10−2	3.11 × 10−3
MaF2	CMGBO	2.29 × 10−2	3.20 × 10−2	2.63 × 10−2	2.68 × 10−2	2.27 × 10−3
	CMPSO	1.34 × 10−1	2.43 × 10−1	1.68 × 10−1	1.69 × 10−1	2.49 × 10−2
	CMODE	7.59 × 10−1	9.96 × 10−1	9.02 × 10−1	8.87 × 10−1	6.45 × 10−2
	MOCLPSO	5.36 × 10−1	9.61 × 10−1	7.44 × 10−1	7.56 × 10−1	9.22 × 10−2
	MOEA/D-DE	3.11 × 10−1	5.39 × 10−1	4.32 × 10−1	4.27 × 10−1	6.21 × 10−2
MaF3	CMGBO	4.91 × 10−2	4.67 × 10−3	1.17 × 10−3	1.64 × 10−3	1.54 × 10−3
	CMPSO	1.15 × 10−3	3.37 × 10−4	6.83 × 10−3	9.82 × 10−3	8.67 × 10−3
	CMODE	3.23 × 10−3	8.00 × 10−3	5.98 × 10−3	5.86 × 10−3	1.17 × 10−3
	MOCLPSO	1.86 × 10−2	4.75 × 10−3	2.98 × 10−3	2.96 × 10−3	1.11 × 10−3
	MOEA/D-DE	2.42 × 10−2	1.05 × 10−4	3.03 × 10−3	3.41 × 10−3	2.36 × 10−3
MaF4	CMGBO	6.94 × 10−2	6.32 × 10−1	1.12 × 10−1	1.92 × 10−1	1.97 × 10−1
	CMPSO	1.92 × 10−1	6.65 × 10−1	3.41 × 10−1	3.54 × 10−1	1.07 × 10−1
	CMODE	3.00 × 10−1	1.10 × 10−2	8.15 × 10−1	7.67 × 10−1	2.64 × 10−1
	MOCLPSO	8.34 × 100	7.94 × 10−1	6.00 × 10−1	5.39 × 10−1	1.83 × 10−1
	MOEA/D-DE	7.07 × 100	6.57 × 10+1	2.82 × 10+1	2.88 × 10+1	1.21 × 10+1
MaF5	CMGBO	3.64 × 10−2	5.48 × 10−2	4.23 × 10−2	4.33 × 10−2	4.17 × 10−3
	CMPSO	5.60 × 10−2	2.59 × 10−1	7.49 × 10−2	1.22 × 10−1	8.36 × 10−2
	CMODE	5.90 × 10−2	9.66 × 10−2	7.65 × 10−2	7.60 × 10−2	1.03 × 10−2
	MOCLPSO	1.25 × 10−1	3.58 × 10−1	2.51 × 10−1	2.47 × 10−1	6.77 × 10−2
	MOEA/D-DE	4.44 × 10−2	5.96 × 10−1	1.07 × 10−1	2.11 × 10−1	2.12 × 10−1
MaF6	CMGBO	7.40 × 10−4	1.14 × 10−3	8.59 × 10−4	8.70 × 10−4	9.54 × 10−5
	CMPSO	1.45 × 10−3	9.78 × 10−3	3.89 × 10−3	4.22 × 10−3	2.18 × 10−3
	CMODE	3.23 × 10−3	8.18 × 10−2	4.25 × 10−2	3.80 × 10−2	1.83 × 10−2
	MOCLPSO	5.54 × 10−3	2.52 × 10−2	1.39 × 10−2	1.47 × 10−2	4.38 × 10−3
	MOEA/D-DE	9.76 × 10−4	2.69 × 10−3	1.23 × 10−3	1.30 × 10−3	3.21 × 10−4

. For the MaF1 and MaF2 test functions, the CMPSO algorithm outperforms the other algorithms. Our approach comes directly after. For the Maf3, MaF4, MaF5 and MaF6 test functions, the CMGBO algorithm dominates its competitors, which proves the high competitiveness of our approach with respect to convergence for this set of test functions. Good convergence of the proposed CMGBO algorithm can be clearly observed in Figure 8, Figure 9 and Figure 10. On the other hand, the results for the HV metric are listed in

Table 6. Statistical results for HV on MaF test functions.

Algorithm		Best	Worst	Median	Average	Std.
MaF1	CMGBO	1.72 × 10−1	1.27 × 10−1	1.54 × 10−1	1.54 × 10−1	8.73 × 10−3
	CMPSO	1.97 × 10−1	1.80 × 10−1	1.90 × 10−1	1.89 × 10−1	4.11 × 10−3
	CMODE	7.53 × 10−2	3.15 × 10−2	4.59 × 10−2	4.73 × 10−2	9.39 × 10−3
	MOCLPSO	1.15 × 10−1	4.52 × 10−2	7.33 × 10−2	7.46 × 10−2	1.59 × 10−2
	MOEA/D-DE	1.43 × 10−1	1.15 × 10−1	1.33 × 10−1	1.31 × 10−1	7.37 × 10−3
MaF2	CMGBO	1.77 × 10−1	1.34 × 10−1	1.56 × 10−1	1.56 × 10−1	8.71 × 10−3
	CMPSO	1.99 × 100	1.81 × 100	1.92 × 100	1.92 × 100	4.35 × 10−2
	CMODE	6.93 × 10−1	5.57 × 10−1	5.86 × 10−1	5.90 × 10−1	2.75 × 10−3
	MOCLPSO	9.38 × 10−1	5.10 × 10−1	6.40 × 10−1	6.53 × 10−1	1.02 × 10−1
	MOEA/D-DE	1.33 × 100	1.12 × 100	1.25 × 100	1.25 × 100	5.75 × 10−2
MaF3	CMGBO	4.91 × 10−1	0	0	1.58 × 10−2	8.82 × 10−2
	CMPSO	0	0	0	0	0
	CMODE	0	0	0	0	0
	MOCLPSO	0	0	0	0	0
	MOEA/D-DE	0	0	0	0	0
MaF4	CMGBO	3.94 × 10−1	0	0	3.64 × 10−2	9.08 × 10−2
	CMPSO	0	0	0	0	0
	CMODE	1.70 × 10−1	0	0	5.48 × 10−2	3.05 × 10−2
	MOCLPSO	0	0	0	0	0
	MOEA/D-DE	0	0	0	0	0
MaF5	CMGBO	5.25 × 10−1	4.87 × 10−1	5.12 × 10−1	5.10 × 10−1	1.07 × 10−2
	CMPSO	4.87 × 10−1	3.07 × 10−1	4.43 × 10−1	4.16 × 10−1	6.26 × 10−2
	CMODE	4.47 × 10−1	3.89 × 10−1	4.26 × 10−1	4.22 × 10−1	1.60 × 10−2
	MOCLPSO	3.78 × 10−1	8.75 × 10−2	1.67 × 10−1	1.89 × 10−1	8.09 × 10−2
	MOEA/D-DE	5.19 × 10−1	9.09 × 10−2	4.36 × 10−1	3.65 × 10−1	1.53 × 10−1
MaF6	CMGBO	1.99 × 10−1	1.93 × 10−1	1.98 × 10−1	1.97 × 10−1	1.64 × 10−3
	CMPSO	1.96 × 10−1	1.38 × 10−1	1.90 × 10−1	1.87 × 10−1	1.22 × 10−2
	CMODE	1.66 × 10−1	0	9.27 × 10−2	8.76 × 10−2	4.38 × 10−2
	MOCLPSO	1.88 × 10−1	1.09 × 10−1	1.42 × 10−1	1.47 × 10−1	1.74 × 10−2
	MOEA/D-DE	1.97 × 10−1	1.77 × 10−1	1.95 × 10−1	1.93 × 10−1	4.91 × 10−3

. It is evident that CMGBO obtains the best statistical results in most cases, which means that the proposed algorithm benefits from a good equilibrium between convergence and diversity compared to the selected methods.

For the Wilcoxon test, the results are presented in

Table 7. The p-values of the Wilcoxon test for the IGD metric.

CMGBO VS
	CMPSO	CMODE	MOCLPSO	MOPSO
MaF1	1.00 × 100	7.01 × 10−12	7.01 × 10−12	7.01 × 10−12
MaF2	1.00 × 100	7.01 × 10−12	7.01 × 10−12	7.72 × 10−12
MaF3	5.46 × 10−9	3.46 × 10−4	3.28 × 10−4	6.64 × 10−4
MaF4	3.46 × 10−4	3.60 × 10−9	6.00 × 10−8	7.15 × 10−3
MaF5	7.01 × 10−12	7.01 × 10−12	7.01 × 10−12	3.45 × 10−11
MaF6	7.01 × 10−12	7.01 × 10−12	7.01 × 10−12	8.99 × 10−11

and

Table 8. The p-values of the Wilcoxon test for the HV metric.

CMGBO VS
	CMPSO	CMODE	MOCLPSO	MOPSO
MaF1	1.00 × 100	7.01 × 10−12	7.01 × 10−12	5.67 × 10−11
MaF2	1.00 × 100	7.01 × 10−12	7.01 × 10−12	7.01 × 10−12
MaF3	1.67 × 10−1	1.67 × 10−1	1.67 × 10−1	1.67 × 10−1
MaF4	5.57 × 10−3	2.15 × 10−2	5.57 × 10−3	5.57 × 10−3
MaF5	7.72 × 10−12	7.01 × 10−12	7.01 × 10−12	5.33 × 10−9
MaF6	2.04 × 10−10	7.01 × 10−12	7.01 × 10−12	2.50 × 10−7

, and we can clearly notice that, in almost all the adopted test functions, the p-values of the CMGBO algorithm compared to all the selected algorithms are less than 1%. These results suggest that the proposed CMGBO algorithm significantly outperforms its competitors with a significance level greater than 99%.

4.5. Discussion

CMGBO, which was designed in this paper, uses as many swarms as the objective number and lets each swarm focus on optimizing one objective. These swarms work cooperatively and communicate through an external hared archive. CMGBO benefits from the following three aspects when solving MOPs:

(1)

As each swarm focuses on optimizing one objective, it can use conventional or any other improved GBO to solve a single-objective problem. Importantly, the difficulty of fitness assignment can be avoided;

(2)

As an external shared archive is used to store the non-dominated solutions found by different swarms, and because the shared-archive information is used to guide the seed update, the algorithm can use the whole search information to approximate the whole PF quickly;

(3)

As ε-dominance is performed on the archived solutions in the update process, the algorithm is able to avoid local PFs. This is helpful for MOPs with multimodal objective functions or with complicated Pareto sets.

The performance of the proposed CMGBO was tested on different sets of MOPs with various objective functions, PFs and Pareto sets. CMGBO was compared with some state-of-the-art and modern MOEAs and MOPSOs, based on IGD and HV. The experimental results show that CMGBO not only generally outperforms the compared algorithms on ZDT problems, but it also generally performs remarkably better than all the other algorithms on DTLZ problems. When dealing with UF problems with complicated Pareto sets, CMGBO is also one of the most promising algorithms. The experimental results support the good performance of CMGBO on these metrics. Furthermore, the benefit of the shared archive used in CMGBO was investigated to demonstrate both the benefit of ε-dominance in bringing in diversity to avoid local PFs and the benefit of the archived information in guiding the particles to approximate the PF quickly. Lastly, the results indicate that the MPMO technique may contribute to reducing the search complexity for each swarm, and a small population size is efficient enough to obtain good performance.

The CMGBO algorithm proposed in this paper is an instantiation of the MPMO technique by using CMGBO. However, as it is a new and general technique using multiple populations to tackle multiple objectives in MOPs, there are still many interesting issues worth further studying in MPMO. Future research work will include the following aspects:

(1)

Applying other algorithms and their improved variants to realize MPMO for the further evaluation of the performance of the multiple-population algorithms in solving MOP. Nevertheless, the information-sharing and population communication strategy should be redesigned when new optimization algorithms are used. For example, we can let individuals perform crossover among different populations for GA information sharing and communication;

(2)

Using the MPMO-based multi-objective algorithms to solve MOPs with many objectives and to solve those problems in dynamic, noisy and uncertain environments;

(3)

Applying the new MPMO-based multi-objective algorithms to real-world problems.

5. Conclusions

A new multi-objective algorithm was proposed by investigating the recent garden balsam optimization method. To achieve this goal, the ε-dominance relation was adopted as an archiving technique, and the notion of an external archive was used to store the Pareto solutions achieved thus far. Moreover, an effective leader selection procedure derived from crowding distance computations was employed to lead the proposed CMGBO algorithm toward a well-distributed PF.

To prove the efficiency of the CMGBO, 15 benchmark functions of varying difficulty were used and compared with 4 powerful and recent algorithms. In the field of convergence and diversity, the proposed method demonstrated ideal and even superior consequences to the selected methods. For future research, the algorithm is recommended to be employed to discrete optimization problems, such as the feature selection problem.