A Novel Scheme of Control Chart Patterns Recognition in Autocorrelated Processes

Abstract

Control chart pattern recognition (CCPR) can quickly recognize anomalies in charts, making it an important tool for narrowing the search scope of abnormal causes. Most studies assume that the observations are normal, independent and identically distributed (NIID), while the assumption of independence cannot always be satisfied under continuous manufacturing processes. Recent research has considered the existence of autocorrelation, but the recognition rate is overestimated. In this paper, a novel scheme is proposed to recognize control chart patterns (CCPs) in which the inherent noise is autocorrelated. By assuming that the inherent noise follows a first-order autoregressive (AR (1)) process, the one-dimensional convolutional neural network (1DCNN) is applied for extracting features in the proposed scheme, while the grey-wolf-optimizer-based support vector machine (GWOSVM) is employed as a classifier. The simulation results reveal that the proposed scheme can effectively identify seven types of CCPs. The overall accuracy is 89.02% for all the autoregressive coefficients, and the highest accuracy is 99.43% when the autoregressive coefficient is on the interval (−0.3, 0]. Comparative experiments indicate that the proposed scheme has great potential to identify CCPs in autocorrelated processes.

1. Introduction

The control chart is an essential tool to monitor whether the manufacturing process is in control or not. There are several key practical efficiencies of using the control chart. First, it is an effective tool for improving the stability of manufacturing processes [1,2]. Second, it provides indicative information which can effectively prevent potential defects [3]. Third, the information on manufacturing system ability is provided [4].

As first proposed by Shewhart, a control chart is composed of two elements: three straight lines parallel to the horizontal axis and sampled observations in chronological order [5]. The traditional control chart is widely used, but there are still some limitations: (1) Analysis methods based on traditional judgment rules cannot point out all abnormal situations. (2) It only focuses on whether the observations are within limits or not, and it cannot provide potential information of previous observations. (3) It cannot provide more clues of abnormal situations, so it is difficult to find out assignable causes. With the development of statistical process control (SPC) and computer technology, the appearance of control chart patterns (CCPs) overcomes these limitations of the traditional control chart. The CCP is composed of continuous points that reflect fluctuations in manufacturing processes [6]. As such, the application of a CCP is exceptionally useful for a rapid diagnosis of any abnormal causal pattern and then for the formulation of a treatment scheme.

Fifteen types of CCPs are introduced in the Statistical Quality Control Handbook [7]. The normal pattern (NOR), cycle pattern (CYC), upward trend pattern (UT), downward trend pattern (DT), upward shift pattern (US), downward shift pattern (DS) and system pattern (SYS) are basic patterns, while the remaining types are combinations of a single pattern and some other special patterns. Since each type of abnormal pattern corresponds to a particular assignable cause [8], control chart pattern recognition (CCPR) dramatically reduces the search scope for abnormal causes. In practice, the type of CCP always corresponds to some specific assignable causes. Thus, CCPR can quickly recognize anomalies in charts, making it an important tool for narrowing the search scope of abnormal causes. The workload of inspectors will be significantly decreased. Developing the CCPR scheme is important for finding out the assignable cause.

In the past, most studies have concentrated on improving the recognition accuracy of CCPs. The rule-based expert system method was originally proposed by Alexander and Jagannathan, who proved that the CCP could be explained and analyzed according to artificial experience [9]. With this breakthrough, excessive false alarms emerged as a critical defect of the system. Subsequently, despite the abundance of research into these problems, recognition accuracy remained low [10]. With the development of computer technology, machine-learning algorithms have been employed in the CCPR field. For example, artificial neural networks [11], support vector machine (SVM) [12,13,14], random forest [15], decision tree [16,17], fuzzy systems [12] and other algorithms have performed well for CCPR. As a result, recognition accuracy improved dramatically so that a multilayer perceptual neural network identified six types of CCPs with an accuracy of 99.15% [18]. Ranaee, Ebrahimzadeh and Ghaderi [19] applied an improved particle swarm algorithm to optimize SVM, and the recognition accuracy is 99.58%. Kalteh and Babouei [20] suggested an adaptive neuro-fuzzy inference system recognition method based on the intelligent application of shape and statistical features. Meanwhile, a chaotic whale optimization algorithm was used to optimize every layer of the classifier, allowing the recognition accuracy to reach 99.77%.

In the literature, most of the existing research assumes that observations at different time points are normal, independent and identically distributed (NIID). Unfortunately, this assumption cannot satisfy flow industries [21], as for instance, there could be autocorrelation in chemical, pharmaceutical or metallurgical industries. Moreover, with the development of sensor technology, more and more advanced acquisition systems have been employed in production to collect process data [22]. Due to the acquisition method of high-frequency data, the time interval is extremely short between adjacent observations, and as a result, there is an inevitable autocorrelation [23].

If the NIID-based CCPR models are directly employed to monitor the autocorrelated process, there will be a great number of false alarms. At present, there are several studies considering autocorrelated processes in CCPR. Cheng and Cheng [24] used a ruled-based neural network as a classifier to identify five types of abnormal CCPs in autocorrelated processes and employed the Haar discrete wavelet transform for decorrelation and feature extraction. Lin, Guh and Shiue [25] proposed a CCPR model based on SVM for the online recognition of seven abnormal CCPs. The simulation results indicated that the method based on SVM had a better performance compared with the learning vector quantization (LVQ) network. In another study, Yang and Zhou [26] developed an integrated neural network based on the LVQ network. Every individual backpropagation network was trained to identify each pattern with the specific autoregressive coefficient, and the outputs of all backpropagation networks were combined by the LVQ network. In these papers, the process mean was assumed to be the first-order autoregressive (AR(1)) process, and the discrete autoregressive coefficients (−0.9, …, 0.9) were selected to represent various autocorrelated levels. Shao et al. [27] proposed a two-stage framework to recognize seven kinds of both single and concurrent CCPs by assuming the autocorrelated processes, and the accurate identification rate is employed to evaluate the performance. De and Pham [21] first considered that the inherent noise was expressed by the AR(1) model and applied a pattern generation scheme (PGS) to generate datasets. Then, the CCPR model based on the NIID assumption was used to recognize CCPs in autocorrelated processes. Compared with the NO-PGS and PGS datasets, it becomes clear that the former scheme has great recognition accuracy. However, given that a host of samples were discarded when PGS was applied to generate CCP samples, this recognition rate is overestimated.

The existing researches on CCPR have established the autocorrelated model in two ways: one assumes that the process mean follows an AR(1) model, and the other assumes that the inherent noise follows an AR(1) model. If both the process mean and the inherent noise obey the AR(1) model, the two models are equivalent when the process mean is set to zero. In other instances, when the process mean is described by the AR(1) model, there will be a remarkable deviation that can be monitored and recognized easily. For this reason, it is important to identify CCPs in tiny autocorrelated processes by assuming that the inherent noise follows AR(1).

In this paper, a novel scheme is proposed to recognize the control chart patterns in autocorrelated processes, in which the one-dimensional convolutional neural network (1DCNN) is utilized to extract features, and the grey-wolf-optimizer-based support vector machine (GWOSVM) is used as the classifier. The novel scheme provides a very appealing option for the CCPR at a wide range of autocorrelation levels and different types of patterns. Experiments show that the proposed scheme is comparable for some levels of autocorrelation and better for overall accuracy. The rest of this study is organized as follows. Section 2 introduces the basic concepts comprising the convolutional neural network (CNN) and SVM algorithms. Section 3 describes the proposed scheme, and Section 4 contains a series of experimental results and discussions. Finally, Section 5 presents the conclusion.

2. Basic Concepts

2.1. Convolutional Neural Network

CNN is a deep feedforward neural network, consisting of an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer. The convolution and pooling layers are mainly responsible for the feature extraction, selection and optimization. Compared with traditional machine learning, CNN’s advantage is its automatic feature extraction capability [28].

The convolution layer is an essential layer, in which the convolution kernel is used to carry on the convolution operation. A convolution kernel can extract a feature map, so multiple convolution kernels can acquire multiple feature maps. The convolution operation is expressed as follows:

$$x_j^l=f\left({\displaystyle \sum _{i=1}^{G_{l-1}}{x}_i^{l-1}\ast k_{ij}^l+a_j^l}\right)$$

(1)

where $l$ is the sequential amount of the current network layer, $f$ represents an activation function, $G$ represents the number of feature maps, $\ast$ indicates the convolution operation, $k_{ij}^l$ represents a convolution kernel, $x_l^j$ is the jth output feature map and $a_j^l$ represents the additive bias.

The most commonly used activation functions are the rectified linear unit function (ReLU), Sigmoid function and Hyperbolic Tangent function (Tanh). As discussed in Reference [28], the ReLU function is appropriate for the 1DCNN. This paper adopts a ReLU function given by the following:

$$f(x)=\max (0,x)$$

(2)

After the convolution operation, the size of the feature map also changes, which is calculated by the following:

$$H_l\times W_l=[\left(H_l-h\right)/m+1]\times [\left(W_l-w\right)/m+1]$$

(3)

where $H_l$ and $W_l$ represent the height and width, $h$ represents the height of the convolutional kernel, $w$ represents the width of the convolutional kernel, $m$ represents the moving stride and $m$ is set to 1 in this paper. Because 1DCNN is applied, $W_l$ and $w$ are also set to 1.

The pooling layer is simpler than the complex convolution layer, mainly decreasing the dimensions of features and improving calculation speed. There are two common pooling methods [26], max pooling and average pooling, and the latter is adopted in this paper. The average pooling operation tends to retain the overall information, expressed as follows:

$$x_j^l=down\left(x_j^{l-1}\right),j=1,2,3,\cdots ,G_l$$

(4)

where $G_l$ indicates the number of input feature maps, and $down$ is a pooling function.

After the pooling operation, the size of the output feature map also changes, which is calculated by the following:

$$H_l\times W_l=\left(H_{l-1}/n\right)\times \left(W_{l-1}/n\right)$$

(5)

where $n$ represents the step size of the pooling operation and is set to 2 in this paper.

A fully connected layer can link the feature maps extracted through a series of convolution and pooling operations. The equation is given by the following:

$${\mathit{x}}_i=f({\mathit{\omega }}_0{\mathit{f}}_v+{\mathit{b}}_0)$$

(6)

where ${\mathit{f}}_v$ represents the input vector, and ${\mathit{b}}_0$ and ${\mathit{\omega }}_0$ are the bias vector and the weight matrix, respectively.

In the fully connected layer, $M$ features are gained, expressed by the following:

$$M=H_{l-1}\times W_{l-1}\times G_{l-1}$$

(7)

where $M$ represents the number of features, and $G_{l-1}$ means the number of input feature maps of the lth fully connected layer.

In the training process, the backpropagation algorithm is utilized to optimize the weights and biases ($k_{ij}^l$, $a_j^l$, ${\mathit{\omega }}_0$ and ${\mathit{b}}_0$) to minimize the value of the loss function. The mean square error (MSE) is suitable for processing time series [29], and this paper chooses the MSE loss function, given as follows:

$$E(w,b)=\frac{{{\displaystyle \sum _{i=1}^n\left(y-y^{\prime }\right)}}^2}{n}$$

(8)

where $y$ represents the real value, and $y^{\prime }$ is the predicted value.

2.2. Support Vector Machine

SVM is an important classification algorithm in machine learning with the advantages of simplicity, prediction and generalization [30]. Establishing an optimal hyperplane is the main idea, which maximizes the distance between the two classes of samples and the hyperplane [31]. The schematic diagram of the SVM is shown in Figure 1.

SVM is widely applied to solve binary classification problems. The process of binary classification is given as follows.

For a binary classification problem, suppose that the training samples are $\left\{\left(x_1,y_1\right),\left(x_2,y_2\right),\cdots ,\left(x_n,y_n\right)\right\}$, where $x_i\in R^n$, $y_i\in \left\{+1,-1\right\}$. The samples are divided into two categories through a hyperplane according to the sample features. The expression of hyperplane is given by the following:

$${\omega }^{T}x+b=0$$

(9)

where $\omega$ is the normal vector of the hyperplane, and $b$ indicates a relative bias value.

For non-separable samples in linear space, a kernel function is used to map the input feature vector from the raw space to a higher-dimensional space. Common kernel functions include the Radial Basis Function (RBF), Laplace, Bessel and Tanh. This study chose to use the RBF, which is described by the following:

$$k\left(X,Y\right)=\exp (-\vartheta ||X-Y|{|}^2)$$

(10)

where the function $k\left(X,Y\right)$ is a non-negative monotonic function for any fixed $\vartheta >0$.

The hyperplane of high-dimensional space is given by the following:

$$f(x)={\omega }^T\phi \left(x\right)+b={\displaystyle \sum _{i=1}^m{\vartheta }_i}{y}_{i}k\left(X,Y\right)+b$$

(11)

The hyperplane vector that separates samples into two classes is called the “support vector” of a binary classification problem. To obtain the optimal classification hyperplane, the distance must be maximized between two types of support vectors and the hyperplane.

The distance can be expressed by the following:

$$\tau =\max \frac{2}{\left|\right|\omega \left|\right|}\mathrm{or}\tau =\min \frac{\left|\right|\omega |{|}^2}{2}$$

(12)

Additionally, a slack variable, ${\epsilon }_i$, is introduced to obtain the SVM.

$$\min \frac{\left|\right|\omega |{|}^2}{2}+C{\displaystyle \sum _{i=1}^n{\epsilon }_i},{\epsilon }_i\ge 0$$

(13)

where $C$ represents the penalty coefficient.

3. The Proposed 1DCNN-GWOSVM Scheme

A novel scheme to recognize CCPs is proposed for autocorrelated processes, and Figure 2 shows the flowchart. In detail, the 1DCNN is employed to extract features from autocorrelated CCP inputs, while the SVM optimized by the GWO algorithm is utilized as the classifier to recognize which type the input CCP belongs to. For the sake of simplicity, this paper refers to the proposed scheme simply as the 1DCNN-GWOSVM scheme. The proposed scheme is implemented in three stages.

Stage 1: Data Generation. As discussed in one review [32], due to the lack of fully documented public databases, 41 out of 44 papers evaluated the performance of the CCPR models by simulated data, while only 3 papers implemented real data in their studies. In this study, a large number of CCP samples are generated by the Monte-Carlo method, which is usually applied to generate CCPs both for training and testing. In this study, the process mean, inherent noise and abnormal disturbances components were utilized to generate the data points for the seven CCPs [21]. The specific expressions are given by the following:

$$Y_t=\mu +W_t+G_t$$

(14)

where $Y_t$ represents the observation at time, $t$; $\mu$ indicates the process mean; $W_t$ represents the inherent noise at time, $t$; and $G_t$ represents an abnormal disturbance.

The inherent noise is assumed to follow the AR(1) model, which is given by the following:

$$W_t=\alpha W_{t-1}+{\tau }_t$$

(15)

where $W_{t-1}$ indicates the inherent noise at time, $t-1$; $\alpha$ is autoregressive coefficient; ${\tau }_t$ is a white noise and obeys the standard normal distribution; and ${\tau }_t\sim N\left(0,1\right)$.

Based on Equations (1) and (2), the expressions of different typical CCPs are defined below.

The NOR pattern is expressed by the following:

$$Y_t=\mu +W_t$$

(16)

The CYC pattern is expressed by the following:

$$Y_t=\mu +W_t+{\gamma }_1\times \sin \left(\frac{2\pi t}{{\gamma }_2}\right)$$

(17)

where ${\gamma }_1$ and ${\gamma }_2$ represent the amplitude and period of a cycle, respectively.

The UT and DT patterns are expressed by the following:

$$Y_t=\mu +W_t\pm {\gamma }_3\times t$$

(18)

where ${\gamma }_3$ is the slope of a trend, the UT pattern employs the mark “$+$” and the DT pattern applies the mark “$-$”.

The US and DS patterns are expressed by the following:

$$Y_t=\mu +W_t\pm s\times {\gamma }_4$$

(19)

where $s$ represents the shift position, ${\gamma }_4$ indicates the shift magnitude, the US pattern generated employs the mark “$+$” and the DS pattern applies the mark “$-$”.

The SYS pattern is expressed by the following:

$$Y_t=\mu +W_t+{\gamma }_5\times {\left(-1\right)}^t$$

(20)

where ${\gamma }_5$ represents the systematic departure.

In this study, eight datasets at different autocorrelation levels were generated to validate the performance of the proposed 1DCNN-GWOSVM scheme.

Stage 2: Extract features. The CCPs generated from Stage 1 served as the inputs of the 1DCNN model. Two convolution layers and two pooling layers were alternatively used to select, extract and optimize the features of CCPs. The fully connected layer was used to expand and splice together the features that were extracted from the convolution and pooling layers. The ReLU activation function was used in the convolution layer, while the average pooling function was utilized in the pooling layer. In order to make the extracted features more prominent, the weights, biases and parameters of the convolution and pooling layers were optimized by the backpropagation algorithm. The extracted features were then output in the fully connected layer. The structure diagram of the 1DCNN feature extraction is shown in Figure 3.

Stage 3: Build the GWOSVM classifier. The SVM is employed as the classifier, and the features extracted from Stage 2 serve as the inputs. For the SVM classifier, the penalty term and the kernel function parameter g have a great impact on its classification performance. The GWO algorithm [33] is a new intelligence optimization algorithm inspired by the predatory activity of grey wolves, such as hunting, encircling and attacking their prey. Its advantages are its strong convergence, its few parameters and its readiness for implementation, accomplished through its simulation of social hierarchy and predatory behavior. Therefore, the parameters of SVM are optimized by the GWO algorithm to gain the optimal parameters, abbreviated as the GWOSVM classifier.

The proposed 1DCNN-GWOSVM scheme aims to recognize a CCP and identify its type in actual manufacturing processes. Identifying the type of abnormal CCP greatly narrows the search scope of the abnormal cause. The type of each CCP corresponds to possible causes [20,34], which are shown in

Table 1. The type of CCPs and the possible causes.

The Type of CCPs	Possible Causes
CYC	The change of system environment, the fluctuation of power generation equipment and the rotation of operators periodically.
UT/DT	Fatigue operation, tools wear and equipment performance degradation.
US/DS	The failure of equipment parts, the transformations of raw materials, the machine setting and the introduction of new workers.
SYS	Difference between test sets; the output is checked in the rotation from different production lines.

.

4. Experimental Results and Discussions

To verify the availability and practicability of the proposed method, a series of simulation experiments are conducted in this section. In this study, the experimental data were simulated based on the Monte-Carlo method. All of the experiments were coded in MATLAB and implemented on a personal computer with a 2.50 GHz CPU and 8 GB memory. The package LibSVM in MATLAB was employed to classify the CCP patterns. The recognition accuracy of the test samples was used as the evaluation criterion, and the recognition accuracy of each pattern was intuitively represented by the confusion matrix.

4.1. Simulating CCP Samples in Autocorrelated Processes

According to the generation process and mathematical expression of CCPs outlined in Section 3, many samples are generated to validate the performance of the proposed scheme. This paper considers the CCP samples generated at different autocorrelation levels, including the autoregressive coefficients $\alpha \in \left(-1,\left.-0.7\right]\right.$, $\alpha \in \left(-0.7,\left.-0.5\right]\right.$, $\alpha \in \left(-0.5,\left.-0.3\right]\right.$, $\alpha \in \left(-0.3,\left.0\right]\right.$, $\alpha \in \left(0,\left.0.3\right]\right.$, $\alpha \in \left(0.3,\left.0.5\right]\right.$, $\alpha \in \left(0.5,\left.0.7\right]\right.$ and $\alpha \in \left(0.7,\left.1\right]\right.$.

The actual manufacturing process is a complex system. To ensure the facticity of the simulated samples, the parameters of CCPs are set in a large range, as described in Reference [21]. Without the loss of generality, μ is set to 0. There is also no abnormal disturbance in the NOR pattern. The parameters and equations of seven CCPs are given in

Table 2. The parameters of seven CCPs.

Pattern	Window Length	Equations	Parameter Value
NOR	N = 60	$Y_t=\mu +W_t$	$\mu$ = 0
CYC		$Y_t=\mu +W_t+{\gamma }_1\times \sin \left(\frac{2\pi t}{{\gamma }_2}\right)$	$0.1\le {\gamma }_1\le 3.0$
			$3\le {\gamma }_2\le 16$
UT		$Y_t=\mu +W_t+{\gamma }_3\times t$	$0.01\le {\gamma }_3\le 0.3$
DT		$Y_t=\mu +W_t-{\gamma }_3\times t$	$0.01\le {\gamma }_3\le 0.3$
US		$Y_t=\mu +W_t+s\times {\gamma }_4$	$0.1\le {\gamma }_4\le 3.0$
DS		$Y_t=\mu +W_t-s\times {\gamma }_4$	$0.1\le {\gamma }_4\le 3.0$
SYS		$Y_t=\mu +W_t+{\gamma }_5\times {\left(-1\right)}^t$	$0.1\le {\gamma }_5\le 3.0$

, and assume the parameters follow uniform distribution. The shift point is randomly selected between 15 and 45, while the observation window length is 60. Figure 4 shows the seven types of CCPs. Training a 1DCNN model requires the use of many samples to avoid overfitting, and in this study, a total of 49,000 samples (42,000 for training samples; 7000 for testing samples) were generated to construct training and testing datasets.

4.2. The 1DCNN Feature Extraction

To ensure the availability of the 1DCNN feature extraction, it is necessary to train an integral 1DCNN. The 1DCNN structure adopted herein is similar to the classic LeNet5 model. We conducted a number of simulation experiments to determine the 1DCNN model’s optimal structure parameters, which are given in

Table 3. Structural parameters of the 1DCNN.

Parameters	Specific Value
Number and size of the input layer	$G_1\times H_1\times W_1=1\times 60\times 1$
Size of the first convolutional kernel	$h_2\times w_2=11\times 1$
Number and size of the first convolution layer feature map	$G_2\times H_2\times W_2=7\times 50\times 1$
Number and size of the first pooling layer subsample map	$G_3\times H_3\times W_3=7\times 25\times 1$
Size of the second convolutional kernel	$h_4\times w_4=18\times 1$
Number and size of the second convolution layer feature map	$G_4\times H_4\times W_4=14\times 8\times 1$
Number and size of the second pooling layer subsample map	$G_5\times H_5\times W_5=14\times 4\times 1$
Number of features	$M=56$

.

Firstly, the input of 1DCNN is time-series data. Since the output of the univariate time series is also a univariate time series after convolution calculation, the first convolution layer applies seven convolution kernels, also known as filters, to extract features. The average pooling function is used to further extract the features and reduce the data complexity. Then, the convolution and pooling layers are used to further extract features. Finally, the fully connected layer is used to link the extracted features into one-dimensional feature vectors. In this paper, to show the effect of the 1DCNN feature extraction, we utilize the t-SNE algorithm to visualize raw data and extracted features, as shown in Figure 5 and Figure 6. Comparing Figure 5 with Figure 6, it can be observed that the overlap degree of the seven CCPs is very high in Figure 5. Using the 1DCNN model for feature extraction is more conducive to the classification of CCPs, as the overlap degree of various CCPs has significantly decreased, as shown in Figure 6. It shows that the 1DCNN model performs well in the feature extraction.

4.3. Training and Testing GWOSVM

After extracting features from the raw data, seven CCPs are classified by SVM. The extracted features are used to train and test the SVM classifier. Selecting reasonable parameters is helpful for the SVM classifier to enhance recognition accuracy, so this study used the GWO algorithm to select the optimal parameters.

Initially, 5600 feature vectors (800 samples for each pattern) were employed to train SVM. The GWO algorithm was applied to optimize the penalty factor and kernel function parameter, and to ultimately select the optimal value. The parameters of the GWO algorithm are given in

Table 4. Parameter setting of the GWO optimization algorithm.

Algorithm	Group	Iteration	$\mathit{c}$	$\mathit{g}$
GWO	20	50	[0.01, 100]	[0.01, 100]

. Then, 700 feature vectors (100 samples for each pattern) were applied to test the performance of the GWOSVM classifier. To obtain steady test accuracy, the tenfold cross-validation technique is used in this paper, with an average of 10 results used as the recognition accuracy.

4.4. Results

The recognition performance of the proposed scheme was investigated at different α levels. The confusion matrix [35] was employed to intuitively express the recognition effect. Its diagonal values indicate the correct classification of diverse CCPs, and the other values indicate the percentage of the wrong classification. The confusion matrices of different $\alpha$ levels are shown in Figure 7.

From Figure 7d, it can be found that the total accuracy is 99.43% at an autocorrelation level of $-0.3<\alpha \le 0$. The accuracy of the DT pattern is 96%, and the other 4% is wrongly divided into the DS pattern, with the recognition accuracy of the rest of the CCPs reaching 100%. Figure 7h shows that the lowest accuracy is 75.14% at a high autocorrelation level of $0.7<\alpha <1$. The US and DS patterns have the lowest accuracy.

To show the performance of the proposed scheme in terms of α levels, the overall recognition accuracy is present in

Table 5. The overall accuracy considering all the $\alpha$ levels.

CCP	Overall	NOR	CYC	UT	DT	US	DS	SYS
Accuracy	89.02%	87.24%	85.25%	95.25%	92.88%	85.25%	88.25%	89.38%

, which reveals that the overall accuracy is 89.02% considering all φ levels. The UT and DT trend patterns have higher recognition accuracies of 95.25% and 92.88%, respectively. The US pattern yields the lowest recognition accuracy.

4.5. Discussions

The study on the CCPR in autocorrelated processes is worthwhile. Therefore, to verify the availability of the proposed method, five experiments were conducted.

Table 6. Some details of input and classifier of six experiments.

No.	Input	Classifier
Experiment 1	Raw data	SVM
Experiment 2	7 statistical features	SVM
Experiment 3	7 statistical features and 7 shape features	SVM
Experiment 4	Features extracted by the 1DCNN	SVM
Experiment 5	Features extracted by the 1DCNN	Softmax
The proposed scheme	Features extracted by the 1DCNN	GWOSVM

gives some details of the input and classifier of six experiments in which statistical features (mean, standard deviation, skewness, kurtosis, mean-square value, median and range) [1] and shape features (SB, ACLPI, PSMLSC, AASL, REVE, SRANGE and ABDPE) [36,37] are utilized.

Table 7. The recognition effects of the proposed scheme and other experiments at different α levels.

	α	−1 < α ≤ −0.7	−0.7 < α ≤ −0.5	−0.5 < α ≤ −0.3	−0.3 < α ≤ 0
No.
Experiment 1		61.57%	79.86%	83.14%	83.86%
Experiment 2		59.29%	72.71%	74%	75%
Experiment 3		68.29%	73.43%	79%	78.71%
Experiment 4		78.43%	82%	85.14%	85.29%
Experiment 5		88.47%	89.10%	89.57%	91.20%
The proposed scheme		90.57%	88.71%	93.29%	99.43%
		0 < α ≤ 0.3	0.3 < α ≤ 0.5	0.5 < α ≤ 0.7	0.7 < α < 1
Experiment 1		84.86%	80.71%	73.71%	52.43%
Experiment 2		76.57%	70.14%	64.57%	46.86%
Experiment 3		79%	74.71%	58.86%	50.43%
Experiment 4		83.71%	80.57%	76.57%	67.57%
Experiment 5		88.37%	84.10%	82.66%	73.34%
The proposed scheme		93.43%	87.86%	83.71%	75.14%

gives the recognition effects of the proposed scheme and other experiments at different α levels. Comparing Experiments 1–4, it can be found that different features have a remarkable influence on the recognition results. The features extracted by the 1DCNN can better express the features of CCPs, and this paper demonstrates the 1DCNN model’s high performance regarding feature extraction. When comparing the results of Experiment 4, Experiment 5 and the proposed scheme, it can be proved that the GWOSVM classifier is better than the traditional Softmax and SVM classifiers.

Table 8. Comparison of the overall accuracy.

Comparison	Datasets	Overall Accuracy
Experiment 1	NO-PGS	75.02%
Experiment 2		67.39%
Experiment 3		70.30%
Experiment 4		79.91%
Experiment 5		85.85%
Reference [21]	PGS	90.03%
Reference [21]	NO-PGS	82.54%
The proposed scheme		89.02%

gives the overall accuracy of five experimental methods, the literature and the proposed scheme considering all the α levels. The results show that the recognition accuracy of the proposed scheme is 89.02%, which is a suboptimal performance likely caused by having different datasets in the respective training and testing models. As mentioned in De and Pham [21], datasets are generated by PGS and NO-PGS methods, with different datasets used to train and test the model. Because the PGS method discards at least a quarter of the initial patterns, the range of samples is artificially narrowed compared with the samples generated by NO-PGS. They show that 24.26%, 35.96% and 44.43% of the samples are discarded, while the corresponding recognition accuracies are 89.52%, 89.52% and 90.03%, respectively. When NO-PGS datasets are used to train and test the same model, the recognition accuracy is only 82.54%. This indicates that the difference is too significant to be ignored between the PGS and NO-PGS datasets. In this work, the NO-PGS datasets are used to train and test the 1DCNN-GWOSVM scheme, and the recognition accuracy is 89.02%, a recognition accuracy clearly higher than other methods trained and tested by the NO-PGS datasets. Thus, the proposed scheme has more advantages in recognizing a diverse range of patterns.

5. Conclusions

In this paper, a novel scheme based on the 1DCNN and GWOSVM was proposed to recognize seven types of CCPs in autocorrelated processes, in which the 1DCNN model was applied to extract important features, and GWOSVM was used as a classifier. The initial datasets were generated through the Monte-Carlo method, and then a series of experiments were conducted to prove the validity and correctness of the proposed scheme. The experiment results show that the overall accuracy is 89.02%, considering all the α levels, proving the proposed scheme’s good performance. The comparison experiments indicate that the proposed method is superior to the other five methods and, in fact, has a wider generality and applicable range compared to the findings in the literature.

In summary, the following conclusions can be drawn:

(1)

The 1DCNN is a more effective method to obtain optimal features than one in which features are extracted manually.

(2)

The GWO algorithm is an effective method to enhance the recognition accuracy of the SVM classifier.

(3)

In contrast to the Softmax classifier, the GWOSVM classifier performs well in terms of recognition accuracy.

(4)

The scheme based on 1DCNN feature extraction and GWOSVM classification has a reliable recognition ability at different autocorrelation levels.

(5)

Because this proposed scheme is limited to univariate statistics in autocorrelated processes, developing multivariate SPC is a direction for future work.