A Novel Adaptive Kernel Picture Fuzzy C-Means Clustering Algorithm Based on Grey Wolf Optimizer Algorithm

Abstract

Over the years, research on fuzzy clustering algorithms has attracted the attention of many researchers, and they have been applied to various areas, such as image segmentation and data clustering. Various fuzzy clustering algorithms have been put forward based on the initial Fuzzy C-Means clustering (FCM) with Euclidean distance. However, the existing fuzzy clustering approaches ignore two problems. Firstly, clustering algorithms based on Euclidean distance have a high error rate, and are more sensitive to noise and outliers. Secondly, the parameters of the fuzzy clustering algorithms are hard to determine. In practice, they are often determined by the user’s experience, which results in poor performance of the clustering algorithm. Therefore, considering the above deficiencies, this paper proposes a novel fuzzy clustering algorithm by combining the Gaussian kernel function and Grey Wolf Optimizer (GWO), called Kernel-based Picture Fuzzy C-Means clustering with Grey Wolf Optimizer (KPFCM-GWO). In KPFCM-GWO, the Gaussian kernel function is used as a symmetrical measure of distance between data points and cluster centers, and the GWO is utilized to determine the parameter values of PFCM. To verify the validity of KPFCM-GWO, a comparative study was conducted. The experimental results indicate that KPFCM-GWO outperforms other clustering methods, and the improvement of KPFCM-GWO is mainly attributed to the combination of the Gaussian kernel function and the parameter optimization capability of the GWO. What is more, the paper applies KPFCM-GWO to analyzes the value of an airline’s customers, and five levels of customer categories are defined.

1. Introduction

Clustering analysis is performed to classify a group of objects into different categories, in which objects in the same cluster will be more similar than object in other clusters [1]. It is often used to discover the structure of data that is invisible. According to the range of membership, clustering algorithms can be divided into two categories: hard clustering and soft clustering. Particularly, Fuzzy C-Means clustering (FCM) is a typical center-based soft clustering algorithm [2]. It determines which class the data belongs to by comparing the membership degree, $u_{ij}$, of the data points to each clustering center. The value of the membership degree is determined by the distance or similarity of the data points to the cluster center.

The FCM approach has been used in various fields, such as face recognition [3], intrusion detection [4], customer segmentation [5], image segmentation [6], and so forth. Not satisfied with the clustering effect of the original FCM method, numerous scholars have proposed improved methods to enhance its effectiveness. The improvement of the algorithm has three main directions, optimization of distance, selection of parameters or initial clustering centers, and introduction of other fuzzy sets. To address the problem of noise sensitivity deficiency of the FCM method in dealing with large datasets, Nikhil and Kuhu proposed a novel fuzzy clustering method called Probabilistic Fuzzy C-Means (PFCM) clustering, which combined the probabilistic c-means algorithm with the FCM method [7]. Stelios and Vassilios presented a Fuzzy Local Information C-Means clustering algorithm (FLICM) by considering the local spatial information and grey information to improve the accuracy of image segmentation [8]. Another improvement that has received attention is the introduction of other fuzzy sets into the FCM algorithm to make the information representation in the data more complete, and thus to improve the clustering effect [9]. Hwang and Rhee introduced the interval type-2 fuzzy sets into the FCM algorithm, and constructed the Interval Type-2 Fuzzy C-Means clustering method (IT2FCM) [10]. This method utilizes two fuzzy parameters ($m_1$ and $m_2$) to control the fuzziness of the final clusters. Since the computation time of the IT2FCM method is excessive, some researchers have turned their attention to Intuitionistic Fuzzy Sets (IFS). Xu and Wu put forward the intuitionistic fuzzy c-means clustering approach [11]. Subsequently, a hesitant fuzzy c-means clustering method was raised, and a novel fuzzification and defuzzification method of image information was defined [12]. Hou proposed a novel proximity measure to substitute the traditional distance/similarity measure in the IFCM algorithm, then introduced the Genetic Algorithm (GA) to select the optimal parameters [13].

Considering the neutral degree of decision-makers, the Picture Fuzzy Set (PFS) was proposed based on the IFS [14]. A picture fuzzy set-based approach can be used in situations that contain more ideas or emotions, and it provides more uncertain information and improves clustering accuracy in clustering algorithms. Therefore, many clustering methods based on the Picture Fuzzy C-Means clustering algorithm (PFCM) have been developed by scholars. Considering that the traditional distributed fuzzy clustering algorithm requires huge computational cost with suboptimal effect, Son [15] proposed a new Distributed Picture Fuzzy C-Means clustering algorithm (DPFCM) via the concept of PFS. Then, Thong and Son summarized the conventional Picture Fuzzy C-Means clustering algorithm model (FC-PFS) on the basis of DPFCM. Subsequently, a comparative study showed that the clustering effect of the FC-PFS algorithm is significantly better than other related fuzzy clustering algorithms [9]. After that, many improved methods were put forward, such as AFC-PFC [16], PFCM-CD [17], and AEWPFCM [18].

Even though the picture fuzzy clustering algorithm performs well, there are still two problems that need to be solved. First of all, the traditional FCM method, including PFCM, utilizes Euclidean distances between data points and cluster centers. However, relevant experimental results signify that Euclidean distance for clustering algorithms has a high error rate, and is more sensitive to noise and outliers [9]. Furthermore, all features in the Euclidean metric contribute equally to the clustering results. The other issue is that there are numerous parameter settings involved in the PFCM algorithm, and the selection of parameters directly affects the final clustering effect. In the manual setting, it may be difficult to find the most suitable parameter values, and may require different parameter values when processing different datasets.

For the first problem, considering that kernel function has stronger distance representation ability, researchers constructed clustering algorithms combining kernel function with FCM. For example, Graves demonstrated that the Kernel-based FCM method (KFCM) outperformed the traditional FCM algorithm, and that the accuracy of clustering is improved, by implementing a comparative study [19]. However, there is a non-negligible problem that the performance of the KFCM algorithm is sensitive to the values of the parameters taken in the kernel function. In addition, Zhou and Zhang put forward a novel kernel-based IFCM algorithm, and also adopted the State Transition Algorithm (STA) for obtaining the initial cluster centers [20]. Wu and Cao introduced the entropy-like kernel function into the FCM algorithm, with local information constraints [21].

For the second problem, applying an intelligent optimization algorithm is a good choice. For example, the GA is able to convert the process of problem solving into a process similar to the crossover and mutation of chromosomal genes in biological evolution. It can be used to select the optimal parameters of the FCM algorithm, as conducted by Lin [22] and Chou [23]. Moreover, the particle swarm algorithm (PSO) is also very popular for solving this issue. Thong obtained the suitable cluster number, cluster center, and membership by combining PSO with PFCM [16]. Then, Zhang presented a novel FCM-ELPSO for solving the problem that PSO-based clustering methods have poor execution times [24]. Beyond that, other intelligent optimization algorithms are often used in parameter selection problems, such as Ant Colony Optimization (ACO) [25], Artificial Bee Colony (ABC) [26], and Grey Wolf Optimizer (GWO) [27].

Considering the advantages of kernel functions and intelligent optimization algorithms, this paper proposed a Kernel-based Picture Fuzzy C-Means clustering with Grey Wolf Optimizer (KPFCM-GWO) for solving the two abovementioned issues simultaneously. KPFCM-GWO can not only better represent the distance between two sample points, but also identify suitable parameter values for enhancing the performance of the proposed method, according to the characteristics of datasets.

The remainder of this paper is structured as follows: The relative background knowledge of basic concepts and operators associated with fuzzy clustering algorithms and PFSs are introduced briefly in Section 2. The proposed method combining kernel functions with GWO is presented in Section 3. Section 4 completes two experiments using KPFCM-GWO in the context of a machine learning dataset and practical case dataset. Finally, Section 5 presents the conclusion of this paper, and offers some recommendations for further research regarding PFCM.

2. Preliminaries

2.1. Fuzzy C-Means Clustering Algorithm

Differing from traditional hard clustering algorithms, fuzzy clustering algorithms use a fuzzy matrix, $u$, with $N$ rows and $C$ columns to demonstrate the likelihood that each data point belongs to each cluster. The fuzzy matrix, $u$, consists of a group of elements, $u_{kj}$, which represent the membership degree of the $k$th data point belonging to the $j$th cluster. The membership degree, $u_{kj}$ (where $u_{kj}\in \left[0,1\right]$ and $\sum _{j=1}^C{u}_{kj}=1$), is usually measured by distance or similarity. The mathematical model of the FCM algorithm is defined as follows [2]:

$$Min:J=\sum _{k=1}^N\sum _{j=1}^C{u}_{kj}^m\Vert X_k-V_j\Vert {}^2$$

(1)

where:

• $N$ indicates the number of data points. Each data point is represented in $d$ dimensions;
• $C$ indicates the number of clusters of the dataset that need to be set up in advance ($2\le C\le N$);
• $m$ is the value of fuzzifier degree. A small $m$ results in larger memberships, $u_{kj}$. The value of m generally depends on human knowledge and experience, $m$ is commonly set to 2;
• $u_{kj}$ is the membership degree of the data point $X_k$ belonging to the cluster $C_j$;
• $X=\left\{X_1,X_2,\cdots ,X_N\right\}$ is the dataset in the feature space;
• $V=\left\{V_1,V_2,\cdots ,V_C\right\}$ is the set of cluster centers, where each element, $V_j$, is the center of cluster $C_j$ in $d$ dimensions;
• $\Vert .\Vert$ denotes the Euclidean distance measure.

Applying the Lagrangian method to solve the above equations, the membership degrees, $u_{kj}$, of the affiliation matrix and clustering centers, $V_j$, are obtained as follows:

$$V_j=\frac{{\sum }_{k=1}^N{u}_{kj}^m{X}_k}{{\sum }_{k=1}^N{u}_{kj}^m}$$

(2)

$$u_{kj}=\frac{1}{{\sum }_{i=1}^C{\left(\frac{\Vert X_k-V_j\Vert }{X_k-V_i}\right)}^{\frac{2}{m-1}}}$$

(3)

Since FCM has better local search ability and faster convergence speed compared to a traditional hard clustering algorithm, it is widely applied in many practical cases. However, the fuzzy clustering algorithm also has some defects that cannot be ignored. The use of Euclidean measures in clustering algorithms has a high error rate. Additionally, the choice of parameters can also have an impact on the effectiveness of the algorithm, and improper parameters tend to trap the algorithm in local minimum. For reducing the limitations and enhancing the accuracy of the algorithm, it is required to consider how to choose the appropriate algorithm parameters, the distance/similarity measure, and the initial clustering center determination, etc.

2.2. Picture Fuzzy C-Means Clustering Algorithm

Definition 1

[14].A picture fuzzy set in a nonempty set X is,

$$A=\{⟨x,{\mu }_A\left(x\right),{\eta }_A\left(x\right),{\gamma }_A\left(x\right)⟩|x\in X\}$$

(4)

where${\mu }_A\left(x\right)$,${\eta }_A\left(x\right)$, and${\gamma }_A\left(x\right)$indicate the positive degree, neutral degree, and negative degree of data point x, respectively. Moreover, these three values satisfy the following constraints:

$$\begin{gathered} {\mu }_A\left(x\right),{\eta }_A\left(x\right),{\gamma }_A\left(x\right)\in \left[0,1\right], \\ 0\le {\mu }_A\left(x\right)+{\eta }_A\left(x\right)+{\gamma }_A\left(x\right)\le 1, \end{gathered}$$

Thus, the refusal degree could be measured as ${\xi }_A\left(x\right)=1-\left({\mu }_A\left(x\right)+{\eta }_A\left(x\right)+{\gamma }_A\left(x\right)\right)$. Using the Yager intuitionistic fuzzy complement function [28], the refusal degree is also described as ${\xi }_A\left(x\right)=1-\left({\mu }_A\left(x\right)+{\eta }_A\left(x\right)\right)-{\left(1-{\left({\mu }_A\left(x\right)+{\eta }_A\left(x\right)\right)}^{\alpha }\right)}^{\frac{1}{\alpha }}$. The parameter $\alpha \in \left(0.1\right]$ is an exponent coefficient that is commonly set to 0.5.

Combining the original FCM algorithm with PFS, Son put forward a novel fuzzy clustering method, named the Picture Fuzzy C-Means clustering algorithm (PFCM) [15]. On the basis of the mathematical model of FCM, the mathematical model of PFCM is adjusted as follows:

$$Min:J=\sum _{k=1}^N\sum _{j=1}^C{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m\Vert X_k-V_j\Vert {}^2+\sum _{k=1}^N\sum _{j=1}^C{\eta }_{kj}\left(\log {\eta }_{kj}+{\xi }_{kj}\right)$$

(5)

The constraints are defined as follows:

$$\{\begin{array}{c}{u}_{kj}+{\eta }_{kj}+{\xi }_{kj}\le 1,\\ {\displaystyle \sum _{j=1}^C}\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)=1\\ {\displaystyle \sum _{j=1}^C}\left({\eta }_{kj}+\frac{{\xi }_{kj}}{C}\right)=1\end{array}$$

(6)

Applying the Lagrangian method to determine the optimal solution of the above model (6) and (7) results in the subsequent equations:

$${\xi }_{kj}=1-\left(u_{kj}+{\eta }_{kj}\right)-{\left(1-{\left(u_{kj}+{\eta }_{kj}\right)}^{\alpha }\right)}^{\frac{1}{\alpha }}$$

(7)

$$u_{kj}=\frac{1}{{\sum }_{i=1}^C\left(2-{\xi }_{kj}\right){\left(\frac{\Vert X_k-V_j\Vert }{\Vert X_k-V_i\Vert }\right)}^{\frac{2}{m-1}}}$$

(8)

$${\eta }_{kj}=\frac{e^{-{\xi }_{kj}}}{{\sum }_{i=1}^C{e}^{-{\xi }_{ki}}}\left(1-\frac{1}{C}\sum _{i=1}^C{\xi }_{ki}\right)$$

(9)

$$V_j=\frac{{\sum }_{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m{X}_k}{{\sum }_{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m}$$

(10)

The computational process of the PFCM method is illustrated in Algorithm 1.

Algorithm 1 PFCM algorithm

I: Dataset $X$ with number of data points ($N$) in $d$ dimensions; number of clusters ($C$); threshold $\epsilon$; fuzzifier $m$; exponent $\alpha$, and the maximal number of iterations $maxSteps>0$ O: Matrices $u, \eta , \xi$ and the cluster centers, $V$.

1: $t=0$, t is the number of current iterations; 2: $u_{kj}^{\left(t\right)} \leftarrow$ random; ${\eta }_{kj}^{\left(t\right)} \leftarrow$ random; ${\xi }_{kj}^{\left(t\right)} \leftarrow$ random satisfying constraint (7); 3: $t=t+1$; 4: Calculate current clustering center $V_j^{\left(t\right)}$ using Equation (10); 5: Update $u_{kj}^{\left(t\right)}$, ${\eta }_{kj}^{\left(t\right)}$, and ${\xi }_{kj}^{\left(t\right)}$ using Equations (7)–(9); 6: Repeat step (4)–step (7); 7: Continue until $\Vert u^{\left(t\right)}-u^{\left(t-1\right)}\Vert +\Vert {\eta }^{\left(t\right)}-{\eta }^{\left(t-1\right)}\Vert +\Vert {\xi }^{\left(t\right)}-{\xi }^{\left(t-1\right)}\Vert \le \epsilon$ or $t>maxSteps$.

Even though PFCM improves upon FCM and IFCM, and enhances the accuracy of the algorithm, there are still two limitations.

Firstly, the value of parameters, such as fuzzifier, $m$, and $\alpha$ of Yager’s generating function, are usually set by users, which depends on their experience and knowledge. However, unsuitable parameter values can lead to lower accuracy and performance of the algorithm. Secondly, the traditional fuzzy clustering algorithm often utilizes the Euclidean distance measure in the objective function to estimate the distance or similarity between data points and cluster centers. It is proven that the use of Euclidean distance in fuzzy clustering algorithms often leads to higher error rates and greater sensitivity to noise and outliers [29].

To cope with the two disadvantages mentioned above, we combine the kernel function and Grey Wolf Optimizer (GWO) to select the optimal parameters.

3. Proposed PFCM Algorithm Based on Kernel Function and GWO

The proposed KPFCM-GWO algorithm is an adaptive fuzzy clustering method that can be split into two parts: (1) Kernel-based Picture Fuzzy C-Means clustering (KPFCM) and (2) parameter selection of KPFCM with GWO.

3.1. Kernel-Based Picture Fuzzy C-Means Clustering (KPFCM)

Over the years, kernel-based approaches have received great attention, and have been used in many fields, such as face recognition [30], image classification [31], forecasting [32], damage detection [33], and so on. The kernel-based approach enables the feature space of the original dimension to be mapped to a higher dimensional feature space through arbitrary nonlinearity using the kernel equations. This feature of the kernel equation could enhance the expressiveness of linear machines, and its use in clustering algorithms can improve their clustering power and accuracy. The following classes of kernel equations are commonly used in algorithms.

• Gaussian kernel function:

  $$K\left(x,y\right)=exp\left(-\frac{\Vert x-y\Vert {}^2}{{\sigma }^2}\right)$$

  (11)
• Linear kernel function:

  $$K\left(x,y\right)=x^{T}y$$

  (12)
• Radial basis kernel function:

  $$K\left(x,y\right)=exp\left(-\frac{{\sum }_i{\left|x_i^a-y_i^a\right|}^b}{{\sigma }^2}\right) \left(0<b\le 2\right)$$

  (13)
• Hyper tangent kernel function:

  $$K\left(x,y\right)=1-\tanh \left(-\frac{\Vert x-y\Vert {}^2}{{\sigma }^2}\right)$$

  (14)

  where $x$ and $y$ indicate two different data points that are mutually independent, and $\sigma$ represents the standard deviation that should be set in advance.

Based on the original PFCM method, the proposed KPFCM algorithm utilized the kernel function to substitute Euclidean distance measure, which is used in the objective function to calculate the distance or similarity between data points and cluster centers. Correspondingly, the mathematical model of the KPFCM algorithm is adjusted as follows:

$$Min:J=\sum _{k=1}^N\sum _{j=1}^C{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m\Vert \varphi \left(X_k\right),\varphi {\left(V_j\right)\Vert }^2+\sum _{k=1}^N\sum _{j=1}^C{\eta }_{kj}\left(\log {\eta }_{kj}+{\xi }_{kj}\right)$$

(15)

where $\Vert \varphi \left(X_k\right),\varphi {\left(V_j\right)\Vert }^2$ indicates the Euclidean distance between two kernel spaces, $\varphi \left(X_k\right)$ and $\varphi \left(V_j\right)$. The two kernel spaces are calculated by Gaussian kernel function using Equation (16). Then, the target function of the above model can be defined as Equation (17).

$$\Vert \varphi \left(X_k\right),\varphi {\left(V_j\right)\Vert }^2=K\left(X_k,X_k\right)+K\left(V_j,V_j\right)-2K\left(X_k,V_j\right)=2\left(1-K\left(X_k,V_j\right)\right)$$

(16)

$$Min:J=2\sum _{k=1}^N\sum _{j=1}^C{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m\left(1-K\left(X_k,V_j\right)\right)+\sum _{k=1}^N\sum _{j=1}^C{\eta }_{kj}\left(\log {\eta }_{kj}+{\xi }_{kj}\right)$$

(17)

The constraints are defined as follows:

$$\{\begin{array}{c}{u}_{kj}+{\eta }_{kj}+{\xi }_{kj}\le 1,\\ {\displaystyle \sum _{j=1}^C}\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)=1\\ {\displaystyle \sum _{j=1}^C}\left({\eta }_{kj}+\frac{{\xi }_{kj}}{C}\right)=1\end{array}$$

(18)

Now, applying the Lagrangian method to calculate the optimal solution of the above model (18) and (19), we get:

$$u_{kj}=\frac{1}{{\sum }_{i=1}^C\left(2-{\xi }_{kj}\right){\left(\frac{1-K\left(X_k,V_j\right)}{1-K\left(X_k,V_i\right)}\right)}^{\frac{1}{m-1}}}$$

(19)

$${\eta }_{kj}=\frac{e^{-{\xi }_{kj}}}{{\sum }_{i=1}^C{e}^{-{\xi }_{ki}}}\left(1-\frac{1}{C}\sum _{i=1}^C{\xi }_{ki}\right)$$

(20)

$${\xi }_{kj}=1-\left(u_{kj}+{\eta }_{kj}\right)-{\left(1-{\left(u_{kj}+{\eta }_{kj}\right)}^{\alpha }\right)}^{\frac{1}{\alpha }}$$

(21)

$$V_j=\frac{{\sum }_{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^{m}K\left(X_k,V_j\right)X_k}{{\sum }_{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^{m}K\left(X_k,V_j\right)}$$

(22)

Proof. 

Taking the deviation of $J$ by $V_j$, we have:

$$\frac{\partial J}{\partial V_j}=4\sum _{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m·\frac{\left(X_k-V_j\right)\Vert X_k-V_j\Vert {}^2}{{\sigma }^4}K\left(X_k,V_j\right)$$

Let $\frac{\partial J}{\partial V_j}=0$, then,

$$\begin{gathered} \sum _{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m\left(X_k-V_j\right)K\left(X_k,V_j\right)=0 \\ {V}_j=\frac{{\sum }_{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^{m}K\left(X_k,V_j\right)X_k}{{\sum }_{k=1}^N{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^{m}K\left(X_k,V_j\right)} \end{gathered}$$

The Lagrangian function with respect to $u$ is,

$$\begin{gathered} L\left(u\right)=2\sum _{k=1}^N\sum _{j=1}^C{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m\left(1-K\left(X_k,V_j\right)\right)+ \\ \sum _{k=1}^N\sum _{j=1}^C{\eta }_{kj}\left(\log {\eta }_{kj}+{\xi }_{kj}\right)-{\lambda }_k\left(\sum _{j=1}^C\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)-1\right) \end{gathered}$$

Let $\frac{\partial L\left(u\right)}{\partial u_{kj}}=0$, then,

$$\begin{gathered} \frac{\partial L\left(u\right)}{\partial u_{kj}}=2m{u}_{kj}^{m-1}{\left(2-{\xi }_{kj}\right)}^m\left(1-K\left(X_k,V_j\right)\right)-{\lambda }_k\left(2-{\xi }_{kj}\right)=0 \\ {u}_{kj}=\frac{1}{2-{\xi }_{kj}}{\left(\frac{{\lambda }_k}{2m\left(1-K\left(X_k,V_j\right)\right)}\right)}^{\frac{1}{m-1}} \end{gathered}$$

In addition, Lagrange’s multiplier method satisfies the following:

$$\sum _{j=1}^C{\left(\frac{{\lambda }_k}{2m\left(1-K\left(X_k,V_j\right)\right)}\right)}^{\frac{1}{m-1}}=1$$

Then,

$${\lambda }_k={\left(\frac{1}{{\sum }_{j=1}^C{\left(2m\left(1-K\left(X_k,V_j\right)\right)\right)}^{\frac{1}{1-m}}}\right)}^{m-1}$$

Plugging ${\lambda }_k$ into $u_{kj}$, we have:

$$u_{kj}=\frac{{\left(1/\left(1-K\left(X_k,V_j\right)\right)\right)}^{\frac{1}{m-1}}}{{\sum }_{i=1}^C\left(2-{\xi }_{kj}\right){\left(1/\left(1-K\left(X_k,V_i\right)\right)\right)}^{\frac{1}{m-1}}}$$

Similarly, the Lagrangian function with respect to $\eta$ is

$$\begin{gathered} L\left(\eta \right)=2\sum _{k=1}^N\sum _{j=1}^C{\left(u_{kj}\left(2-{\xi }_{kj}\right)\right)}^m\left(1-K\left(X_k,V_j\right)\right)+\sum _{k=1}^N\sum _{j=1}^C{\eta }_{kj}\left(\log {\eta }_{kj}+{\xi }_{kj}\right) \\ -{\lambda }_k\left(\sum _{j=1}^C\left({\eta }_{kj}+\frac{{\xi }_{kj}}{C}\right)-1\right) \end{gathered}$$

Let $\frac{\partial L\left(u\right)}{\partial {\eta }_{kj}}=0$, then,

$$\begin{gathered} \log {\eta }_{kj}+1-{\lambda }_k+{\xi }_{kj}=0 \\ {\eta }_{kj}=e^{{\lambda }_k-1-{\xi }_{kj}} \end{gathered}$$

as well as

$$\sum _{j=1}^C\left(e^{{\lambda }_k-1-{\xi }_{kj}}+\frac{{\xi }_{kj}}{C}\right)=1$$

and subsequently

$$e^{{\lambda }_k-1}=\frac{1-\frac{1}{C}{\sum }_{j=1}^C{\xi }_{kj}}{{\sum }_{j=1}^C{e}^{-{\xi }_{kj}}}$$

Therefore,

$${\eta }_{kj}=\frac{e^{-{\xi }_{kj}}}{{\sum }_{i=1}^C{e}^{-{\xi }_{ki}}}\left(1-\frac{1}{C}\sum _{i=1}^C{\xi }_{ki}\right)$$

The proof is complete. □

The algorithm calculates the current cluster center and the affiliation matrix during each iteration, and stops when the threshold condition is met or the maximum number of iterations is reached and outputs the current clustering result. The detailed iterative process of KPFCM algorithm is described in Algorithm 2:

Algorithm 2 KPFCM algorithm

I: Dataset $X$ with number of data points ($N$) in $d$ dimensions; number of clusters ($C$); threshold $\epsilon$; fuzzifier $m$; exponent $\alpha$; kernel function $\sigma$, and the maximal number of iterations $maxSteps>0$ O: Matrices $u, \eta , \xi$ and the cluster centers, $V$.

1: $t=0$; 2: $u_{kj}^{\left(t\right)} \leftarrow$ random; ${\eta }_{kj}^{\left(t\right)} \leftarrow$ random; ${\xi }_{kj}^{\left(t\right)} \leftarrow$ random satisfying constraint (19); 3: $t=t+1$; 4: Calculate $V_j^{\left(t\right)}$ via Equation (22); 5: Calculate $u_{kj}^{\left(t\right)}$, ${\eta }_{kj}^{\left(t\right)}$ and ${\xi }_{kj}^{\left(t\right)}$ via Equations (19)–(21); 6: Repeat; 7: Continue until $\Vert u^{\left(t\right)}-u^{\left(t-1\right)}\Vert +\Vert {\eta }^{\left(t\right)}-{\eta }^{\left(t-1\right)}\Vert +\Vert {\xi }^{\left(t\right)}-{\xi }^{\left(t-1\right)}\Vert \le \epsilon$ or $t>maxSteps$.

3.2. Parameter Selection of KPFCM with GWO

The selection of parameters also has a great impact on the performance of the fuzzy clustering algorithm, such as the fuzzier $m$, iteration number $maxSteps$, and threshold $\epsilon$. As we all know, a suitable parameter can effectively enhance the accuracy of the clustering algorithm. Therefore, we select parameters that will have an impact on the performance of the proposed algorithm to be studied in this paper. It has been suggested that the fuzzifier parameter, $m$, of the IFCM algorithm impacts its clustering effect [34]. Similarly, the parameter $m$ will make a difference to the KPFCM-GWO algorithm. Beyond that, since the proposed method introduces the Gaussian kernel function, compared to other fuzzy clustering algorithms, there is one more parameter, $\sigma$, that needs to be considered. The parameter $\sigma$ of the Gaussian kernel function controls the radial range of action. It has been confirmed that the value of the parameter $\sigma$ of the Gaussian kernel function affects the performance of the algorithm [19]. Therefore, in order to select the optimal parameter values, $m$ and $\sigma$, in the KPFCM algorithm, we utilize one of the intelligent optimization algorithms.

The Grey Wolf Optimization algorithm was proposed by Mirjalili et al. [27], and was inspired by the prey-catching behavior of wolves. The GWO algorithm has numerous benefits, such as few parameters, easy implementation, simple structure, the algorithm is equipped with a convergence factor and information feedback mechanism, and has a good balance between local searches and global searches. It has received wide attention from scholars, and has been successfully applied to various fields in recent years, especially in parameter optimization. Therefore, it is a good choice to introduce this algorithm to determine the parameter values. The mathematical model and procedure of GWO are defined below.

There is a severe set of stratification guidelines in the wolf pack, and the GWO algorithm describes its stratification through a mathematical model. The top three best wolves (optimal solutions) are defined as $\alpha ,\beta$, and $\delta$, respectively, and they guide the other wolves to search toward the target. The remaining wolves (candidate solutions) are defined as ω, and they update their positions around $\alpha ,\beta$, or $\delta$.

• Encircling prey

Grey wolves gradually approach their prey and surround it when they let it go. The mathematical model of a grey wolf encircling its prey ($m$ and $\sigma$) is as follows:

$$\overrightarrow{D}=\left|\overrightarrow{C}·{\overrightarrow{X}}_P\left(t\right)-\overrightarrow{X}\left(t\right)\right|$$

(23)

$$\begin{gathered} \overrightarrow{X}\left(t+1\right)={\overrightarrow{X}}_P\left(t\right)-\overrightarrow{A}·\overrightarrow{D} \\ \overrightarrow{A}=2\overrightarrow{a}·{\overrightarrow{r}}_1-\overrightarrow{a} \\ \overrightarrow{C}=2·{\overrightarrow{r}}_2 \end{gathered}$$

(24)

where $t$ represents the current iterations, $\overrightarrow{A}$ and $\overrightarrow{C}$ are coefficient vectors, ${\overrightarrow{X}}_P$ and $\overrightarrow{X}$ are the position vector of the prey and the position vector of the grey wolf, respectively. Throughout the iterations, $\overrightarrow{a}$ decreases linearly from 2 to 0. ${\overrightarrow{r}}_1$ and ${\overrightarrow{r}}_2$ are the random vectors in (0, 1).

• Hunting prey

The grey wolf, led by the first three wolves $\alpha ,\beta$, and $\delta$, can identify the location of the prey (optimal solution) and approach it step by step. In the algorithmic model, during each round of iteration, the first three wolves search for the location of the prey, and the remaining grey wolves update their own locations based on their location information. This process of finding the location of the prey can be simulated by the following mathematical model:

$$\{\begin{array}{c}{\overrightarrow{D}}_{\alpha }=\left|{\overrightarrow{C}}_1·{\overrightarrow{X}}_{\alpha }-\overrightarrow{X}\right|\\ {\overrightarrow{D}}_{\beta }=\left|{\overrightarrow{C}}_2·{\overrightarrow{X}}_{\beta }-\overrightarrow{X}\right|\\ {\overrightarrow{D}}_{\delta }=\left|{\overrightarrow{C}}_3·{\overrightarrow{X}}_{\delta }-\overrightarrow{X}\right|\end{array}$$

(25)

$$\overrightarrow{X}\left(t+1\right)=\frac{{\overrightarrow{X}}_1+{\overrightarrow{X}}_2+{\overrightarrow{X}}_3}{3}$$

(26)

• Attacking prey

In order to simulate the approach to the prey, the value of $\overrightarrow{a}$ is gradually reduced from 2 to 0, and therefore the fluctuation range of $\overrightarrow{A}$ is also reduced. In other words, as the value of $\overrightarrow{a}$ reduces linearly from 2 to 0 during the iterative process, the corresponding value of $\overrightarrow{A}$ varies within the interval $\left(-a,a\right)$. When $|\overrightarrow{A}|<1$, the next position of the grey wolf can be located anywhere between its current position and the prey’s position. In other words, the wolves attack the prey (fall into local optimum).

• Search for prey

Grey wolves search for prey according to the position of $\alpha ,\beta$, and $\delta$. Based on mathematical modeling of the scatter, random values of $\overrightarrow{A}>1$ or $\overrightarrow{A}<-1$ can be used to force the grey wolf to separate from its prey, which allows the GWO algorithm to globally search for optimal solutions. The GWO algorithm has another component, $\overrightarrow{C}$, which indicates the random weights of the prey effects of the location of the wolf to help discover new solutions. The stochasticity of $\overrightarrow{C}$ plays a very important role in avoiding local optima when the algorithm is trapped in a local optimum and cannot be easily jumped out, especially in the final iteration where the global optimum solution needs to be obtained.

In this study, the KPFCM-GWO clustering algorithm combined PFCM and kernel function, and the GWO method is applied to determine the most suitable value of parameters of the proposed KPFCM to enhance the performance. The procedures of the KPFCM-GWO clustering algorithm are shown in Figure 1.

4. Experiments

4.1. Datasets

This pare performs two different experiments in five datasets. The first four datasets (Iris, Wine, Glass, and Wisconsin Diagnostic Breast Cancer (WDBC)) are benchmark datasets from UCI Machine Learning Repository [35]. The other is the airline customer dataset. The details of each dataset are shown in

Table 1. The description of experimental datasets.

	Datasets	Number of Elements	Number of Features	Number of Classes
Experiment 1	Iris	150	4	3
	Wine	178	13	3
	Glass	214	9	6
	WDBC	569	30	2
Experiment 2	Airline	62,988	25	5

.

The Iris and Wine datasets are often used in experiments on machine learning algorithms, especially for clustering and classification algorithms. The classification of Glasses is performed by the content of different chemical elements. The range of values for each feature is relatively small, and the differentiation is not large, which is a test of the clustering algorithm. The WDBC dataset consists of medical data, which has huge elements and a small number of classes. These datasets have different distinctions. Consequently, the experiment outcomes with these datasets can be convinced. The above four benchmark datasets were utilized to exam the performance and advantages of the proposed algorithm and the specific procedure is shown in Experiment 1. Moreover, a comparison of different fuzzy clustering algorithms on benchmark datasets is also performed in Experiment 1.

In order to show the application of the proposed algorithm to a practical problem, Experiment 2 applies KPFCM-GWO to analyze the value of airline’s customers in Airline. The Airline dataset has 62988 elements and 25 attributes including airline membership information, member flight information and member points information, and it spans two years.

Some evaluation indexes are used to reveal the validity of the KPFCM-GWO algorithm in this paper. In general, there are two criteria to assess the performance of the clustering algorithm, internal quality evaluation metrics and external evaluation metrics. In this paper, four indexes are selected for evaluation.

• Calinski–Harabasz (CH) takes into account the tightness within clusters as well as the separation between clusters. Thereby, a larger CH represents a tighter class itself and a more dispersed class to class, i.e., a better clustering result. The CH index can be formulated as Equation (27).

  $$\begin{gathered} CH\left(K\right)=\frac{\frac{tr\left(B\right)}{\left(K-1\right)}}{\frac{tr\left(W\right)}{\left(N-K\right)}} \\ tr\left(B\right)=\sum _{j=i}^k\Vert z_j-{z\Vert }^2 \\ tr\left(W\right)=\sum _{j=1}^k\sum \Vert x_i-z_j\Vert {}^2 \end{gathered}$$

  (27)
• Silhouette Coefficient (SC) assesses the effectiveness of clustering by combining the cohesiveness and separation of clusters. The value is between −1 and 1. The closer the samples of the same cluster are to each other, and the more distance between the samples of different clusters, the larger the value of SC will be, indicating a better clustering effect.
• The Adjusted Rand Index (ARI) takes values in the range of −1 to 1. The closer the value of ARI is to 1, the closer the clustering result is to the real situation, and the better the clustering effect is.
• The accuracy index refers to the ratio of the number of correctly predicted samples to the total number of predicted samples, and it does not consider whether the predicted samples are positive or negative cases [20].

4.2. Experiment 1

Firstly, we demonstrate the process of the proposed method to cluster the Iris dataset. In this case, $N=150, d=4,C=3, \alpha =0.5, \epsilon =0.0000001, maxSteps=100, SearchAgent{s}_{no}=6, dim=2$. The value of fuzzier $m$ and parameter $\sigma$ are determined by GWO that the range of $m$ is 1–5, the range of $\sigma$ is 1–20. The initial matrices of positive degree, $U$, neutral degree, $\eta$, and refusal degree, $\xi$, are randomly generated between 0 to 1, and the initial cluster center, $V$, are calculated by the original FCM algorithm, which are presented as follows:

$$\begin{gathered} U_{t=0}=\left[\begin{array}{ccc}0.1963& 0.0139& 0.7897\\ 0.4479& 0.3122& 0.2399\\ \cdots & \cdots & \cdots \\ 0.5215& 0.2569& 0.2216\\ 0.7047& 0.2404& 0.0548\end{array}\right] \\ {\eta }_{t=0}=\left[\begin{array}{ccc}0.3141& 0.2419& 0.1013\\ 0.2431& 0.2875& 0.2822\\ \cdots & \cdots & \cdots \\ 0.2006& 0.1891& 0.2844\\ 0.1080& 0.2114& 0.2564\end{array}\right] \\ {\xi }_{t=0}=\left[\begin{array}{ccc}0.4080& 0.4999& 0.1059\\ 0.2805& 0.3494& 0.4009\\ \cdots & \cdots & \cdots \\ 0.2553& 0.4437& 0.4107\\ 0.1776& 0.4407& 0.4933\end{array}\right] \\ {V}_{t=0}=\left[\begin{array}{cccc}5.8389& 3.0193& 3.8047& 1.2251\\ 5.8811& 3.0636& 3.8339& 1.2271\\ 5.7293& 3.0886& 3.5299& 1.0986\end{array}\right] \\ {U}_{t=100}=\left[\begin{array}{ccc}0.0562& 0.0707& 0.8731\\ 0.0851& 0.1078& 0.8071\\ \cdots & \cdots & \cdots \\ 0.5998& 0.3053& 0.0949\\ 0.5042& 0.4174& 0.0784\end{array}\right] \\ {\eta }_{t=100}=\left[\begin{array}{ccc}0.2316& 0.3031& 0.0175\\ 0.1822& 0.3013& 0.0188\\ \cdots & \cdots & \cdots \\ 0.1949& 0.2931& 0.2076\\ 0.3247& 0.2592& 0.3193\end{array}\right] \\ {\xi }_{t=100}=\left[\begin{array}{ccc}0.4973& 0.4752& 0.1062\\ 0.4994& 0.4610& 0.1658\\ \cdots & \cdots & \cdots \\ 0.1935& 0.3503& 0.4950\\ 0.1631& 0.2919& 0.4659\end{array}\right] \\ {V}_{t=100}=\left[\begin{array}{cccc}4.9948& 3.3993& 1.4734& 0.2423\\ 6.4328& 2.9520& 5.2118& 1.8863\\ 5.7281& 2.7119& 4.1356& 1.2724\end{array}\right] \end{gathered}$$

To eliminate interference as much as possible and simulate the real clustering situation, the sequence of the data points in the original Iris dataset is randomly broken up. Its initial distribution is shown in Figure 2. From this figure. We then perform fuzzy clustering on the broken-up dataset for verifying the effect of the KPFCM-GWO algorithm. After performing the proposed method, KPFCM-GWO, the final clustering result is shown in Figure 3. We can observe that the Iris dataset is clustered into three different clusters accurately. Adjusting the value of parameter $C$, and keeping the other parameter values constant, the algorithm is applied to the remaining three public datasets. The values of the corresponding evaluation indexes are shown in

Table 2. The value of evaluation index.

	Accuracy	SC	CH	ARI
Iris	0.93	0.579	558.85	0.745
Wine	0.88	0.563	557.46	0.412
Glass	0.85	0.391	88.39	0.311
WDBC	0.73	0.707	1317.16	0.518

Note: SC—Silhouette Coefficient, CH—Calinski–Harabasz, ARI—Adjusted Rand Index.

.

As shown in Table 2, the same clustering algorithm has different performances on different datasets. In particular, comparing the clustering outcomes of the Iris dataset and the Glass dataset. The Iris dataset has only 4 attributes, while the Glass dataset has up to 9 attributes. The data points of the 9 attributes in the Glass dataset are complex and are not easy to classify correctly. That is to say, fuzzy clustering algorithms do not perform as well on complex and noisy datasets as they do on standard datasets. Besides, it is also worth noting that the value of CH index of the WDBC experiment is the largest of the four datasets. This indicates that the tightness of the classes in the WDBC dataset and the separation between classes is high.

In the following, to verify the validity of KPFCM-GWO, Fuzzy C-Means clustering (FCM) [2], Intuitionistic Fuzzy C-Means clustering (IFCM) [36], Kernel-based Fuzzy C-Means clustering (KFCM) [19], Picture Fuzzy C-Means clustering (PFCM) [9] and Picture Fuzzy C-Means clustering with Particle Swarm Optimization algorithm (PFCM-PSO) [16] are selected for comparative experiment.

Figure 4 presents the clustering performance of the above comparison methods on four UCI datasets. As seen in Figure 4, on the Iris dataset, the IFCM algorithm obtains the best accuracy, while FCM and the proposed method have better CH values than other methods. Combined

Table 3. Clustering performance with different clustering algorithms.

	Accuracy	SC	CH	ARI
FCM
Iris	0.89	0.549	558.99	0.729
Wine	0.86	0.566	559.40	0.354
Glass	0.71	0.258	88.68	0.227
WDBC	0.63	0.697	1300.21	0.491
IFCM
Iris	0.95	0.535	556.34	0.731
Wine	0.83	0.531	317.56	0.373
Glass	0.73	0.277	81.23	0.327
WDBC	0.57	0.711	1267.69	0.535
KFCM
Iris	0.91	0.521	529.618	0.768
Wine	0.69	0.559	554.568	0.366
Glass	0.52	0.280	61.482	0.239
WDBC	0.866	0.691	1273.503	0.529
PFCM
Iris	0.89	0.535	537.786	0.730
Wine	0.702	0.520	528.919	0.371
Glass	0.542	0.261	71.367	0.267
WDBC	0.849	0.704	1266.015	0.476
PFCM-PSO
Iris	0.94	0.562	541.22	0.741
Wine	0.85	0.519	553.87	0.452
Glass	0.78	0.310	40.36	0.296
WDBC	0.72	0.715	1310.26	0.505

Note: SC—Silhouette Coefficient, CH—Calinski–Harabasz, ARI—Adjusted Rand Index. FCM—Fuzzy C-Means clustering, IFCM—Intuitionistic Fuzzy C-Means clustering, KFCM—Kernel-based Fuzzy C-Means clustering, PFCM—Picture Fuzzy C-Means clustering, PFCM-PSO—Picture Fuzzy C-Means clustering with Particle Swarm Optimization algorithm.

with Figure 4 and Figure 5, each clustering algorithm yields different clustering results, and different evaluation factors behave differently on different clustering algorithms. For instance, in addition to the CH index, among the three remaining evaluation factors that take values less than 1, the accuracy for the Iris dataset achieves the maximum value regardless of the clustering algorithm with the range being 0.89–0.95. However, the clustering effect of the algorithm also decreases when clustering those datasets with noise, such as the Glass dataset, where the value of accuracy drops to 0.52–0.79. The same situation exists for the other evaluation factors. For example, the values of ARI for standard dataset such as Iris is higher with the range being 0.73 to 0.77, and other complex and noisy data such as Glass dataset would reduce the ARI ranges to 0.24–0.33. In the case of CH index, the complex and noisy dataset such as the Glass dataset would have a lower value, while the experiment in datasets with significant differences between the data points such as the WDBC dataset has a higher value of CH index.

From the above experimental procedure and comparative analysis, it is clear that the proposed algorithm is generally better than other fuzzy clustering algorithms, although it cannot achieve the best results in all datasets. The improvement in algorithm performance is mainly attributed to the combination of the Gaussian kernel function and the parameter optimization capability of the GWO.

There are also some parameters in the Grey Wolf Optimization algorithm that need to be set in advance. Apparently, as the number of wolves increases, the algorithm will take more time. Hence, does this mean that the more the number of wolves, the better the efficiency of the algorithm? How does one balance time complexity, algorithm efficiency, and the number of wolves? To clear up the above uncertainties, we ran the KPFCM-GWO algorithm with a different number of wolves for the four benchmark datasets, and measured the accuracy and silhouette coefficient of those cases. The results are presented in Figure 6 and Figure 7.

The outcomes in Figure 6 obviously state that a larger number of wolves does not make the algorithm more accurate, and the suitable range of the number of wolves should be between 5 and 8. It is clear that the accuracy values of the KPFCM-GWO method on the four datasets are maximal with 6. When considering another clustering index, silhouette coefficient, it is obvious that changes in the number of wolves did not have a significant effect on SC values. It can be roughly seen that the best range of values for the number of wolves, in this case, is between 5 and 7. When the wolf pack value is taken to be 6, each dataset achieves a high SC value. Therefore, in this paper, the number of wolves takes the value of 6.

4.3. Experiment 2

The increasingly competitive market requires airlines to adapt their marketing strategies to attract more customers to fly with them. An existing airline was facing a serious business crisis, including loss of customers to other competitors, declining competitiveness of the company, resulting in fewer and fewer new members, and waste of resources due to underutilization of the company’s resources. In order to solve these problems and improve the company’s competitiveness, the company intends to establish a set of customer value assessment models to accurately and reasonably classify the airline’s current customers and implement personalized marketing strategies for different types of customers.

Based on the customers’ basic information, flight information and points information in the airline system, a time period with a width of two years was selected as the observation window for analysis based on the last flight date (LAST_FLIGHT_DATE), with 31 March 2014 as the end time, and all customers with flight records from 1 April 2012 to 31 March 2014 were extracted to consist the historical data with a total of 62,988 records (for more detail see

Table 4. Airline membership related information.

	Attributes
Customers’ basic information	MEMBER_NO
	FFP_DATE
	FIRST_FLIGHT_DATE
	GENDER
	FFP_TIER
	WORD_CITY
	WORK_PROVINCE
	WORK_CONUTRY
	AGE
Flight information	FLIGHT_COUNT
	LOAD_TIME
	LAST_TO_END
	AVG_DISCOUNT
	SUM_YR
	SEG_KM_SUM
	LAST_FLIGHT_DATE
	AVG_INTERVAL
	MAX_INTERVAL
Points information in the airline system	EXCHANGE_COUNT
	EP_SUM
	PROMOPTIVE_SUM
	PARTNER_SUM
	POINTS_SUM
	POINT_NOTFLIGHT
	BP_SUM

). In conjunction with the large amount of member profile information the airline has accumulated, and the flight records it has taken, the following objectives are requested:

• Segmentation of customers with the help of airline customer data.
• Characterize different customer categories and compare the value of different categories of customers.

In this experiment, the proposed method KPFCM-GWO is used to solve the customer value classification issue mentioned above.

A model is often used in customer value analysis and identification problems which identifies high-value customers by segmenting customer groups through recent consumption interval (Recency), consumption frequency (Frequency) and consumption amount (Monetary), as shown in Figure 8. The model consists of three elements: Recency, Frequency and Monetary, in short, RFM.

In the airline industry, since airline fares are influenced by all kinds of factors, such as flight distance, demand, seasonal factors, cabin class, and so on, it is unreasonable to measure the value of a customer to an airline based on the amount spent alone, and the traditional RFM model is not appropriate in this application scenario. Therefore, a novel customer value identification model is obtained by adding the customer’s accumulated flight miles $M$ in the time period and the discount factor $C$ corresponding to the class of travel to replace the customer’s spending amount in the original model on the basis of the RFM model. Moreover, the length of membership of airline members can also influence the customer value to a certain extent, so this paper incorporates the length of customer relationship $L$ into the model and obtains the LRFMC model [37]. In summary, this case uses the duration of membership $L$, consumption interval $R$, consumption frequency $F$, flight mileage $M$ and average discount factor $C$ as airline identification customer value indicators, and the specific meaning of each indicator is shown in

Table 5. LRFMC model.

Model	LRFMC
$L$	Length of membership enrollment in the observed time period
$R$	Number of months since the customer’s last flight ended in the observed time period
$F$	The number of times the customer flew with the company during the observed time period counted
$M$	Customer’s accumulated flight miles within the observation window
$C$	Average of the discount factors corresponding to the customer’s class of travel within the observation window

below.

The original data contained 62,988 records with 44 attributes, but there were missing values and redundant or irrelevant attributes that required advanced data preprocessing before they could be used. Therefore, we first deleted the records with 0 fares (SUM_YR = 0) and kept the records with nonzero fares (SUM_YR$\ne$0) or nonzero average discount rates (AVG_DISCOUNT$\ne$0), and total flight kilometers greater than 0 (SEG_KM_SUM$\ne$0). After data cleaning, the data comprised 62,043 records and 44 attributes. Then, based on the LRFMC model, here, six attributes related to the model, FFP_DATE, LOAD_TIME, FLIGHT_COUNT, AVG_DISCOUNT, SEG_KM_SUM, and LAST_TO_END, were selected, and their irrelevant, weakly relevant, or redundant attributes are removed. Finally, due to the large data variation in the range of values of the selected indicators, the dataset is standardized by Z-score [38] in order to eliminate the effects of order of magnitude data, and to better meet the needs of the clustering algorithm.

In this step, applying the proposed method, KPFCM-GWO, to customer data for customer classification and the number of clusters is 5. The clustering result is shown in

Table 6. Airline customer data clustering results.

Cluster	Cluster Center					No. Cluster
	ZL	ZR	ZF	ZM	ZC
Customer 1	0.0593	−0.1977	−0.2353	2.1068	1.7729	15,739
Customer 2	−0.3328	−0.7509	1.3731	0.7999	−0.0403	4182
Customer 3	−0.7618	−0.7053	0.4666	0.7435	0.0125	24,661
Customer 4	0.8262	−0.6901	0.7813	0.8079	−0.1205	12,125
Customer 5	1.2951	−0.8648	2.8262	3.4351	0.4086	5336

Note: ZL—length of membership enrollment in the observed time period after Z-score normalization, ZR—number of months since the customer’s last flight ended in the observed time period after Z-score normalization, ZF—the number of times the customer flew with the company during the observed time period counted after Z-score normalization, ZM—customer’s accumulated flight miles within the observation window after Z-score normalization, ZC—average of the discount factors corresponding to the customer’s class of travel within the observation window after Z-score normalization.

and Figure 9. Feature analysis is performed for the clustering results. Customer group 1 is largest on attribute C and smallest on attribute F. Customer group 3 is smallest on attributes L and M. Customer group 4 is smallest on attribute C. Customer group 5 is largest attribute L, and F and M are smallest attributes for R.

$$\begin{gathered} U_{t=0}=\left[\begin{array}{ccccc}0.2401& 0.2622& 0.1082& 0.1400& 0.2494\\ 0.1276& 0.1019& 0.2493& 0.3438& 0.1773\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ 0.0729& 0.1467& 0.2523& 0.0724& 0.4557\\ 0.3194& 0.0795& 0.3465& 0.1355& 0.1191\end{array}\right] \\ {V}_{t=0}=\left[\begin{array}{ccccc}-0.2911& -0.4253& 0.4238& 0.6219& 0.4025\\ 0.8403& -0.8341& 3.4391& 3.9864& 0.3975\\ 0.6669& -0.5792& 0.9228& 0.8808& 0.4577\\ 0.0426& -0.5021& 0.6334& 0.7522& 0.2676\\ 0.6987& -0.8314& 5.4953& 10.7046& 1.0534\end{array}\right] \\ {U}_{t=100}=\left[\begin{array}{ccccc}0.1704& 0.1276& 0.1255& 0.4619& 0.1145\\ 0.0771& 0.0416& 0.0399& 0.8073& 0.0342\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ 0.2514& 0.3221& 0.3103& 0.0112& 0.1051\\ 0.2823& 0.3982& 0.2032& 0.0123& 0.1037\end{array}\right] \\ {V}_{t=100}=\left[\begin{array}{ccccc}0.0593& -0.1977& -0.2353& 2.1068& 1.7729\\ -0.3328& -0.7509& 1.3731& 0.7999& -0.0403\\ -0.7618& -0.7053& 0.4666& 0.7435& 0.0125\\ 0.8262& -0.6901& 0.7813& 0.8079& -0.1205\\ 1.2951& -0.8648& 2.8264& 3.4351& 0.4086\end{array}\right] \end{gathered}$$

Based on the above description of characteristics, five levels of customer categories are defined: important retention customers, important development customers, important retention customers, average customers, and low-value customers.

• Important retention customers (customer group 1) are high-value customers for airlines, the most desirable type of customer, contributing the most to the airline, yet accounting for a smaller percentage. Airlines should prioritize resources to these customers for differentiation and one-to-one marketing to increase the loyalty and satisfaction of these customers, and to increase the high level of spending of these customers as much as possible.
• Important development customers (customer group 3) are potential value customers for airlines. The airline has to make efforts to encourage such customers to increase their spending on flights with the company and with its partners, to increase their transfer costs to competitors, and to make them gradually become loyal customers of the company.
• Important retention customers (customer group 5) have high uncertainty of changes in customer value. Based on the changes of the recent consumption time and the number of consumptions of these customers, airlines should speculate the variation of customer consumption, and focus on tracking contact with them and adopt certain marketing means to extend the life cycle of customer consumption.
• Average customers and low-value customers (customer group 2 and 4) that they may only fly with the company when airline tickets are on sale at a discount.

5. Conclusions

To solve two problems of the original fuzzy clustering algorithm (i.e., the poor performance of Euclidean distance and the hard selection of parameters), this paper proposed a novel adaptive Kernel-based Picture Fuzzy Clustering algorithm with Grey Wolf Optimizer (KPFPCM-GWO), in which the Gaussian kernel function is used to substitute the Euclidean distance measure, and GWO is introduced to determine the values of the two parameters ($m$ and $\sigma$) of the proposed algorithm. Furthermore, two experiments are conducted on five datasets for verifying the validity of KPFPCM-GWO. The experimental results indicate that the proposed algorithm KPFCM-GWO has good performance, and is generally better than other comparative methods.

However, the proposed algorithm has an important limitation. We applied GWO to select the parameters that improve the clustering validity, but the computation time of the algorithm is also greatly increased. Compared to the increase in time cost, the improvement in algorithmic efficiency is not very significant. Apart from that, the determination of the initial clustering center also has a great influence on the clustering effect of the algorithm. In future research, when improving the fuzzy clustering algorithm, the computation time should be taken into consideration for balancing the improvement of the algorithm performance and the time cost. It is also important to consider how to choose the initial clustering center to achieve very fast clustering and avoid falling into local minimum.