Residual Control Chart Based on a Convolutional Neural Network and Support Vector Regression for Type-I Censored Data with the Weibull Model

Abstract

Control charts with conditional expected value (CEV) can be used with novel statistical techniques to monitor the means of moderately and lowly censored data. In recent years, machine learning and deep learning have been successfully combined with quality technology to solve many process control problems. This paper proposes a residual control chart combining a convolutional neural network (CNN) and support vector regression (SVR) for type-I censored data with the Weibull model. The CEV and exponentially weighted moving average (EWMA) statistics are used to generate training data for the CNN and SVR. The average run length shows that the proposed chart approach outperforms the traditional EWMA CEV chart approach in various shift sizes and censored rates. The proposed chart approach is suitable to be used in detecting small shift size for highly censored data. An illustrative example presents the application of the proposed method in an electronics industry.

1. Introduction

Product service life has always been a quality highly valued by customers. To ensure customers enjoy good reliability, many manufacturers conduct life tests on their products before they leave the factory. As production technology has advanced, products have achieved longer service life. Practitioners thus need to spend increasing time and cost to collect product life data. To save time and cost in testing, practitioners may set a test termination time at which to halt the test, whether or not there has been a failure. As some testing units have not failed by the termination time, practitioners can only obtain incomplete lifetime records. These incomplete lifetime records are termed type-I right-censored data.

Control charts are used to monitor and track process data in order to quickly identify potential variations, which assists practitioners in understanding whether a production process is in an in-control state. As traditional Shewhart-type and exponentially weighted moving average (EWMA)-type control charts cannot be used with right-censored data, novel control charts have been proposed for this purpose [1,2,3]. Steiner and Mackay made multiple proposals, including integrating the $\overline{X}$ control chart with a conditional expected value (CEV) to monitor highly right-censored data and setting a lower control limit (LCL) to monitor lifetime decreases under normally distributed processes, developing Shewhart-type CEV control charts for non-normally distributed right-censored data, and applying an EWMA CEV control chart to increase detection ability for highly right-censored data [4,5,6]. Lee applied an economic design approach to determine the design parameters of the CEV $\overline{X}$ control chart [7].

To detect decreases and increases in average lifetimes under Weibull distributions, Zhang and Chen developed two single-sided EWMA CEV control charts [8]. Subsequently, many researchers proposed EWMA-type CEV and conditional median control charts to control non-normally distributed censored data, achieving superior statistical performance [9,10,11]. Double EWMA (DEWMA) control charts for censored data were developed and it was found that they detect shifts of the scale parameter under gamma distributions more efficiently than EWMA charts [12,13]. Many studies for further applications of DEWMA CEV control charts have also been published [12,14,15]. The EWMA CEV chart has also been modified to monitor various censored data. For example, EWMA CEV control charts were extended to detect average lifetimes for multiple censored data and an EWMA chart was implemented with a modified CEV to monitor Weibull window-censored data [16,17]. In the above-mentioned literature, most of the research focuses on increasing the detection efficiency of moderate and low censored rates and moderate shift sizes [10,12,13,14,15]. Few of the literature have proposed to improve the detection efficiency of small average lifetime shifts and high censored rates using larger sample sizes [3]. Since reliability life testing is a destructive test method, increasing the sample size will lead to increased testing costs. If control charts can improve the detection ability of high censored rates and small shift sizes without increasing sample size, this will help practitioners reduce quality control costs.

With the recent rise of artificial intelligence, many studies have introduced machine-learning or deep-learning methods to solve quality problems. Many researchers have all proposed control charts based on machine or deep learning to improve detection ability [18,19,20,21,22,23,24,25]. Deep-learning methods have achieved excellent performance in failure detection in high-dimensional or complex processes [3,26,27,28]. Moreover, some researchers recognized abnormal control chart patterns using machine- or deep-learning methods [29,30,31,32,33,34]. However, to the authors’ knowledge, there have been no relevant studies that use deep-learning or machine-learning methods to monitor right-censored data.

Convolutional neural networks (CNN) and support vector regression (SVR) are types of deep-learning and machine-learning methods, respectively. The literature indicates that both methods have successfully improved the performance of control charts and enhanced failure detection [28,35,36]. To the authors’ knowledge, however, there has been no study that combines these two methods for process control.

Most existing works have explored various statistical methods to obtain good data features to improve the accuracy of SVR. Finding feature-extraction methods is typically time-consuming, and CNNs have a good ability to extract data features. Thus, combining a CNN and SVR may save time in finding suitable data feature extraction methods and improve prediction accuracy. In addition, the Weibull distribution can present skewed and approximately symmetrical distributions using differently shaped parameter values and is widely used in lifetime analysis. Thus, this study will construct a control chart with a combination of CNN and SVR to improve the detection ability in high censored rates and small shift sizes for Weibull type-I right-censored data.

The remainder of this study is organized as follows. Section 2 presents the basic CNN and SVR methodologies. In Section 3, the proposed method is described, and in Section 4, its performance is investigated. Section 5 provides a real-world case to illustrate the implementation of the proposed chart. Finally, Section 6 summarizes key findings and outlines future development directions.

2. Combined CNN and SVR

2.1. CNN

CNNs are feed-forward neural networks that use multiple convolutional and pooling layers to extract higher-level spatial information from input data. The approach has broad applications, particularly in speech recognition and computer vision [37]. A CNN model generally consists of an input layer, an output layer, and multiple hidden layers, which include convolution layers, pooling layers, and fully connected layers. The sample points plotted on a control chart are similar to one-dimensional sequence data; a control chart is suitable for training the sample points of the control map with a one-dimensional CNN model [28].

In a convolution layer, the output feature map is generated through the convolution of multiple input feature maps and multiple convolutional kernels. The output of a convolution layer is expressed by the following:

$${\zeta }_j^l=f_{RL}\left(b_j^l+{\sum }_{i=1}^{D_{l-1}}{\zeta }_i^{l-1}\times {\omega }_{ij}^{l-1}\right),$$

where $f_{RL}\left(\bullet \right)$ is a rectified linear unit (ReLU) activation function, $l$ is the $l$-th layer of the CNN network, ${\omega }_{ij}^{l-1}$ is the one-dimensional filter kernel from the $i$-th neuron at layer $l-1$ to the $j$-th neuron at layer $l$, $b_j^l$ is the bias of the $j$-th neuron at layer $l$, and $D_{l-1}$ is the number of the input at layer $l-1$.

The pooling layer serves to decrease the feature map’s dimensions and streamline computations. The max pooling method is a commonly implemented pooling operation in this layer. The pooling output at layer $l$ is given by following:

$${Mp}_j^l=\mathrm{m}\mathrm{a}\mathrm{x}\left({\zeta }_j^l\right).$$

A fully connected layer is the last layer of a CNN and follows several convolution and pooling layers. The operation of this layer is similar to a multilayer neural network and the neurons in this layer connect all the activations in the previous layer. The output of a fully connected layer can consider a categorical variable for multi-class classification problems or a regression response variable for relevance prediction problems.

2.2. SVR

SVR can effectively address regression problems involving high-dimensional features by finding a function that approximates the relationship between the input vector and the output values. Let $\left({\mathbf{u}}_1,y_1\right),\dots ,\left({\mathbf{u}}_m,y_m\right)$ be the data set for regression analysis. SVR is then expressed as follows:

$$\begin{gathered} \min \frac{1}{2}{‖\mathbf{w}‖}^2+c_s{\sum }_{i=1}^m\left({\xi }_i+{\widehat{\xi }}_i\right) \\ \mathrm{s}.\mathrm{t}.y_i+{\mathbf{w}}^T{\mathbf{u}}_i-e\le \epsilon +{\xi }_i \\ {\mathbf{w}}^T{\mathbf{u}}_i+e-y_i\le \epsilon +{\xi }_i \\ {\xi }_i,{\widehat{\xi }}_i\ge 0, \end{gathered}$$

(1)

where $‖\bullet ‖$ is the Euclidian norm, $\mathbf{w}$ is the weight vector, $e$ is the bias term, $\epsilon$ is the width of the insensitive zone, ${\xi }_i$ and ${\widehat{\xi }}_i$ are slack variables for relaxing the training errors (where ${\xi }_i,{\widehat{\xi }}_i\ge 0$), and $c_s$ is a positive constant that determines the trade-off between minimizing the error and minimizing the slack variables [9,38].

Next, the dual model of Equation (1) can be established using the Lagrangian method as follows:

$$\begin{array}{l}\max {\sum }_{i=1}^m{y}_i\left({\nu }_i-{\widehat{\nu }}_i\right)-\epsilon {\sum }_{i=1}^m\left({\nu }_i+{\widehat{\nu }}_i\right)-\frac{1}{2}{\sum }_{i=1}^m{\sum }_{j=1}^m\left({\nu }_i-{\widehat{\nu }}_i\right)\left({\nu }_j-{\widehat{\nu }}_j\right){\mathbf{u}}_i^T{\mathbf{u}}_j\\ \mathrm{s}.\mathrm{t}.{\sum }_{i=1}^m\left({\widehat{\nu }}_i-{\nu }_i\right)=0\\ {\widehat{\nu }}_i,{\nu }_i\in \left[0,c_s\right],\end{array}$$

where $\widehat{\nu }$ and ${\nu }_i$ are the Lagrange multipliers and must be non-negative and real. By solving the above, the SVR predictor becomes

$$f_{svr}\left(\mathbf{u},{\widehat{\nu }}_i,{\nu }_i\right)={\sum }_{i=1}^m\left({\widehat{\nu }}_i-{\nu }_i\right)\kappa \left({\mathbf{u}}_i^T,{\mathbf{u}}_j\right)+e,$$

where $\kappa \left({\mathbf{u}}_i^T,{\mathbf{u}}_j\right)$ is a kernel function. The radial basis function (RBF) is a kernel function commonly used in SVR predictors [39].

2.3. Combination of CNN and SVR

In this section, a CNNSVR network model was created that combines CNN and SVR for process control. The next section presents the setup and procedure for a new control chart with the proposed CNNSVR network. In the CNNSVR network, a CNN is trained to generate the feature maps and output the activation values of the last layer, and these activation values are used as the input for the SVR to generate the predicted values, as shown in Figure 1. Because the sample points are one-dimensional sequence data, the CNNSVR network adopts a one-dimensional sequence input layer. A combinational layer is defined as one convolutional layer followed by one pooling layer. Additionally, the ReLU activation function is used in each combinational layer to calculate the output vector. The CNNSVR network consists of N combinational layers for the activation value. This activation value is inputted to the SVR to obtain the predicted value.

In Figure 1, green, brown, and black dashed lines represent the convolution (Co.layer), pooling (Po.layer), and fully connected (FC.layer) layers, respectively. The dark green and orange squares are the kernels of the Co.layer and Po.layer, respectively. Each yellow rectangle and each red rectangle represent a feature map (F.map) of the Co.layer and Po.layer, respectively, the height of the rectangle means the size of the feature map, and K_i is the number of feature maps of i-th layer. The number and size of the convolution kernel (or filter) affects the number and size of the feature map of the convolution layer. The kernel size of the pooling layer determines the size of the feature map of the pooling layer. When using the CNNSVR network, the number of combinational layers and convolution kernels and the size of the convolution and pooling kernels must be determined first to achieve good training and testing results. This paper refers to the literature and uses the trail-error method to optimize these parameter values [18,23].

3. CNNSVR-Based Residual Chart

Deep-learning-based residual control charts have been presented to increase the performance of process control [18,20]. The current paper extends their method to propose a CNNSVR-based residual chart for Weibull type-I right-censored data. The sample point of a control chart in an in-control state is used to train the CNNSVR network to predict the sample point (or statistic) of the next period. In other words, the sample point at period t − 1 is used to predict the sample point at period t. If the predicting sample point is similar to the actual sample point, then this indicates the process is in-control; otherwise, the process mean has shifted. A threshold value is determined by the residual values of the trained CNNSVR network, and it plays a role as the LCL of the traditional control chart to indicate the process state. If the error value is below the threshold value, it indicates a significant difference between the predicted sample point and the actual sample point; therefore, the process is considered to be in an out-of-control state.

The MATLAB R2023a software provides various deep-learning, statistics, and machine-learning toolboxes, which can be easily employed for the simulation processes to investigate the control chart performance.

3.1. EWMA CEV Statistic for a Weibull Distribution

Let $S=\left\{s_1,s_2,\dots ,s_k\right\}$ be a Weibull random variable and $a_0$ and $b_0$ be the scale and shape parameters of an in-control process, respectively. The in-control mean ($M_0$) and variance ($V_0$) are as follows:

$$\begin{gathered} M_0=a_0\mathrm{\Gamma }\left(1+1/b_0\right)\mathrm{and} \\ {V}_0=a_0^2\left\{\mathrm{\Gamma }\left(1+2/b_0\right)-{\left[\mathrm{\Gamma }\left(1+1/b_0\right)\right]}^2\right\}. \end{gathered}$$

The censoring rate for Weibull lifetimes can be expressed as $Pc=exp\left[-{\left(c_T/a\right)}^b\right]$, where $c_T$ is the censoring time. The CEV of the Weibull distribution is as follows:

$$Cev=E\left(S|S\ge c_T\right)=\frac{a_0\mathrm{{\rm Y}}\left(D_w,1+1/b_0\right)}{exp\left(-D_w\right)},$$

where $D_w={\left(c_T/a_0\right)}^{b_0}$ and $\mathrm{{\rm Y}}\left({\theta }_1,{\theta }_2\right)={\int }_0^{{\theta }_1}{r}^{{\theta }_2-1}exp\left(-r\right)dr$ [14].

Practitioners take $n$ samples and measure their lifetime values using the reliability lifetime testing method. Let $s_i$ be the lifetime of the i-th testing sample. Then, the CEV value $x_i$ of the i-th testing sample is as follows:

$$x_i=\left\{\begin{array}{c}{s}_i\hspace{1em}\hspace{1em}for{s}_i\le c_T\\ Cev\hspace{1em}for{s}_i>c_T\end{array}\right..$$

The sample mean of size $n$ is $\overline{X}={\sum }_{i=1}^n{x}_i/n$. In practice, $n=5$ is always used for sampling and plotting control charts. Let $\lambda$ be the smoothing parameter of the EWMA chart. Zhang and Chen [8] showed that the EWMA statistic at period t for monitoring the mean decrease is as follows:

$$E_t=\mathrm{m}\mathrm{i}\mathrm{n}\left\{M_0,\lambda \overline{X_t}+\left(1-\lambda \right)E_{t-1}\right\}$$

(2)

For an EWMA CEV chart used to monitor a process, an LCL is established to detect the mean disease, because practitioners always focus on the detection of average lifetime reduction. The most commonly used criterion to assess control charts’ performance is average run length (ARL) [8,14,16,18,29]. A larger ARL value in an in-control process means a lower false alarm, whereas a smaller ARL value in an out-of-control state represents the faster detection of shifts.

3.2. Setup of the Proposed Control Chart

The main requirement to set up the control chart is to find a reasonable threshold value for the CNNSVR-based residual chart in order to maintain its false alarm rate. To generate large numbers of the EWMA statistic $E_t$ for CNNSVR network training, the following Monte Carlo simulation procedure is applied:

(1)

Generate the lifetimes $s_i$ of n samples from a Weibull distribution with parameters $a_0$ and $b_0$.

(2)

If $s_i>c_T$, then $x_i=CEV$; otherwise, $x_i=s_i$.

(3)

Calculate the sample mean $\overline{X}$.

(4)

Calculate the EWMA statistic $E_t$ using Equation (2).

(5)

Repeat steps (1) to (4) m times to obtain m EWMA statistics $E_t$.

The CNNSVR network is similar to a univariate autoregressive model of order 1 constructed by a neural network, with a record in the CNNSVR training data consisting of an input value and an output value. Let $\left(E_1,E_2\right),\dots ,\left(E_{m-1},E_m\right)$ be the training data set for the CNNSVR network, where $\left(E_{t-1},E_t\right)$ represents a record in the training data, and $E_{t-1}$ and $E_t$ are the input value and output value, respectively, of the CNNSVR network for this record. The input and output vectors have dimensions $1\times \left(m-1\right)$. Increasing m can improve the CNNSVR network’s prediction ability, but this requires longer training time. This study uses m =10,000.

After training, the CNNSVR network outputs the predictive values. The residual value can be obtained using the actual value (the output value of the training data set) minus the predicted value. Next, a reasonable threshold value $\eta$ is determined from these residual values. The threshold value $\eta$ is similar to the LCL of the control charts and affects the ARL value. With consideration for the practitioners’ interest in the state of the reduced product lifetime, only the training residual values $E_t<{\widehat{E}}_t$ are used to determine the threshold value $\eta$. Since $\eta$ is related to the false alarm rate, the practitioner can choose a specific in-control ARL value based on their experience to determine the optimal threshold value ${\eta }^{\mathrm{\ast }}$ with a simulation method. The determination procedure for ${\eta }^{\mathrm{\ast }}$ is shown in Figure 2.

This study used the ‘wblrnd’ function to generate complete Weibull lifetime data for training, and applied the ‘sequenceInputLayer’, ‘convolution1dLayer’, ‘maxPooling1dLayer’, ‘reluLayer’, ‘fullyConnectedLayer’, ‘regressionLayer’, ‘trainNetwork’, ‘activations’, ‘predict’, and ‘fitrsvm’ functions to construct and train the CNNSVR network.

3.3. Performance Simulation for the Proposed Control Chart

In the control phase, the sample statistic at the (t − 1)-st period $E_{t-1}$ is inputted into a trained CNNSVR network to predict the sample statistic ${\widehat{E}}_t$. At the t-th period, the practitioner takes $n$ samples and computes the sample statistic $E_t$. If the prediction error $E_t-{\widehat{E}}_t$ is less than the threshold ${\eta }^{\mathrm{\ast }}$, then the process is identified as being in an out-of-control state; otherwise, the process is in an in-control state.

The occurrence of an assignable cause results in the reduction of the process mean as being in an out-of-control state. Let $M_1=\delta \times M_0$ be the mean of an out-of-control state where the process variance is unchanged and $\delta$ is a mean shift size that can be estimated by $\delta =M_1/M_0$. The Weibull parameters in an out-of-control state can be obtained by solving the following system of simultaneous equations:

$$\left\{\begin{array}{c}{M}_1=a_1\mathrm{\Gamma }\left(1+1/b_1\right)\\ {V_0=a}_1^2\left\{\mathrm{\Gamma }\left(1+2/b_1\right)-{\left[\mathrm{\Gamma }\left(1+1/b_1\right)\right]}^2\right\}\end{array}\right.,$$

where $a_1$ and $b_1$ are the Weibull scale and shape parameters in an out-of-control state, respectively. Out-of-control lifetime data are generated using the Weibull distribution with parameters $a_1$ and $b_1$.

Control charts have different detection efficiencies at different shift sizes. Therefore, the out-of-control ARL value must be evaluated for various possible shift sizes before this control chart is implemented in order to monitor the process. An out-of-control ARL simulation procedure of the CNNSVR-based residual control chart is proposed, as shown in Figure 3. The same MATLAB functions as in Section 3.2 are applied.

4. Performance Comparison

This section compares the ARL performance of the CNNSVR-based residual chart and the EWMA CEV chart. The parameter choice for the Weibull distribution considers the cases of $b_0=0.8,1,2.5,\mathrm{and}4$ and $a_0=1$ for comparison.

Table 1. Skewness of the Weibull distribution.

The Parameters for $\mathbf{Weibull}\mathbf{Distribution}({\mathit{a}}_0$$,{\mathit{b}}_0$)	Process Mean ${\mathit{M}}_0$	Process Standard $\mathbf{Deviation}\sqrt{{\mathit{V}}_0}$	Skewness Coefficient Sk
(1, 0.8)	1.13	1.43	2.81
(1, 1)	1	1	2
(1, 2.5)	0.89	0.38	0.36
(1, 4)	0.91	0.25	−0.09

shows the skewness values (Sk) of the chosen parameter values. The smaller shape parameter values resulted in a greater skewness of the Weibull distribution. For $b_0=4$, the Weibull distribution approximates a symmetric probability distribution. In the comparison, the sample size $n$ is fixed at 5, the smoothing parameter $\lambda$ of the EWMA statistic is 0.1 or 0.2, and $Pc$ is $0.2$, 0.4, and 0.6 for lower and moderate censoring rates [8,13,15]. This paper builds the CNNSVR network with three combinational layers. The maximum epoch of network training is set to 1000 and the size and number of kernels for each combinational layer are as shown in

Table 2. Sizes and numbers of kernels for the CNNSVR network.

Layer			Convolution Layer	Pooling Layer
Combinational Layer	#1	Kernel size	5	1
		Number of kernels	200	-
		Activation function	ReLU
	#2	Kernel size	3	1
		Number of kernels	70	-
		Activation function	ReLU
	#3	Kernel size	3	1
		Number of kernels	50	-
		Activation function	ReLU

. The complete CNNSVR network architecture is shown in Figure 4.

Table 3. Design parameter values of CNNSVR and EWMA charts.

The Parameters for $\mathbf{Weibull}\mathbf{Distribution}({\mathit{a}}_0$$,{\mathit{b}}_0$)	Censored $\mathbf{Rate}\mathit{P}\mathit{c}$	$\mathbf{Smoothing}\mathbf{Parameter}\mathit{\lambda }=0.1$		$\mathbf{Smoothing}\mathbf{Parameter}\mathit{\lambda }=0.2$
		EWMA	CNNSVR	EWMA	CNNSVR
		LCL	${\mathit{\eta }}^{\mathbf{\ast }}$	LCL	${\mathit{\eta }}^{\mathbf{\ast }}$
(1, 0.8)	0.2	0.741	−0.109	0.642	−0.123
	0.4	0.352	−0.048	0.321	−0.072
	0.6	0.122	−0.018	0.112	−0.028
(1, 1)	0.2	0.776	−0.110	0.675	−0.130
	0.4	0.419	−0.045	0.387	−0.077
	0.6	0.168	−0.021	0.156	−0.031
(1, 2.5)	0.2	0.823	−0.059	0.773	−0.088
	0.4	0.712	−0.054	0.679	−0.078
	0.6	0.391	−0.030	0.374	−0.046
(1, 4)	0.2	0.863	−0.040	0.828	−0.068
	0.4	0.837	−0.049	0.806	−0.069
	0.6	0.497	−0.031	0.480	−0.046

shows the values of LCL for the EWMA CEV chart (EWMA chart) and ${\eta }^{\mathrm{\ast }}$ for the CNNSVR-based residual chart (CNNSVR chart), where these values fix their in-control ARL values at 200.

Table 4. ARL values of CNNSVR and EWMA charts for $\lambda =0.1$.

Censored $\mathbf{Rate}\mathit{P}\mathit{c}$	Charts	$\mathbf{Shift}\mathbf{Size}\mathit{\delta }$						$\mathbf{Shift}\mathbf{Size}\mathit{\delta }$
		0.9	0.8	0.7	0.6	0.5	0.2	0.9	0.8	0.7	0.6	0.5	0.2
		(${\mathit{a}}_0,{\mathit{b}}_0$) = (1, 0.8)						(${\mathit{a}}_0,{\mathit{b}}_0$) = (1, 1)
0.2	EWMA	45.8	21.4	13.8	10.3	8.0	5.4	38.5	19.0	9.5	7.2	5.4	3.4
	CNNSVR	71.1#	19.0	13.1	7.9	4.4	2.3	60.5#	18.7	9.2	5.3	3.9	1.4
0.4	EWMA	57.9	35.1	26.7	21.9	18.8	13.4	51.2	30.0	22.0	17.6	14.8	10.2
	CNNSVR	73.6#	34.6	23.0	17.8	14.3	9.4	50.1	24.5	15.2	11.7	9.4	6.3
0.6	EWMA	126.1	79.6	55.5	43.9	36.7	25.5	110.9	67.5	48.0	37.6	30.9	20.7
	CNNSVR	120.9	76.4	49.5	39.0	30.7	20.6	132.0#	66.2	44.8	32.0	25.6	16.1
		($a_0,b_0$) = (1, 2.5)						($a_0,b_0$) = (1, 4)
0.2	EWMA	27.3	9.3	4.6	3.2	2.5	1.3	16.2	4.8	2.6	2.0	1.5	1.0
	CNNSVR	27.2	9.3	3.4	2.2	1.6	1.0	16.1	4.3	1.7	1.2	1.1	1.0
0.4	EWMA	25.2	12.5	8.2	6.1	4.9	3.1	12.1	5.3	3.4	2.6	2.1	1.3
	CNNSVR	29.1#	11.7	6.4	4.3	3.2	1.7	10.3	3.8	2.1	1.3	1.1	1.0
0.6	EWMA	225.6	99.8	45.5	27.8	19.7	10.4	1729.1	922.1	80.7	27.3	17.0	8.1
	CNNSVR	256.3#	89.7	42.1	22.2	15.0	7.0	847.0	534.8	79.9	23.8	12.9	5.1

and

Table 5. ARL values of CNNSVR and EWMA charts for $\lambda =0.2$.

Censored $\mathbf{Rate}\mathit{P}\mathit{c}$	Charts	$\mathbf{Shift}\mathbf{Size}\mathit{\delta }$						$\mathbf{Shift}\mathbf{Size}\mathit{\delta }$
		0.9	0.8	0.7	0.6	0.5	0.2	0.9	0.8	0.7	0.6	0.5	0.2
		$({\mathit{a}}_0,{\mathit{b}}_0$) = (1, 0.8)						$({\mathit{a}}_0,{\mathit{b}}_0$) = (1, 1)
0.2	EWMA	48.6	19.1	10.9	7.4	5.8	3.5	43.4	16.3	8.7	6.0	4.5	2.6
	CNNSVR	44.8	18.2	9.6	4.3	2.6	1.4	42.9	15.0	6.9	4.5	2.8	1.3
0.4	EWMA	49.5	23.0	16.0	12.5	10.3	7.1	45.5	19.6	13.4	10.1	8.3	5.6
	CNNSVR	61.0#	21.7	12.5	7.4	5.9	3.5	50.4#	17.1	10.3	6.3	4.6	2.5
0.6	EWMA	114.5	58.4	34.9	24.8	20.0	12.9	100.8	49.7	30.2	21.3	16.5	10.5
	CNNSVR	114.1	53.4	30.1	18.9	14.3	8.1	94.2	40.4	25.7	15.5	11.5	6.5
		($a_0,b_0$) = (1, 2.5)						($a_0,b_0$) = (1, 4)
0.2	EWMA	28.3	9.4	4.4	3.0	2.2	1.2	15.7	4.5	2.5	1.8	1.3	1.1
	CNNSVR	27.2	9.2	3.7	2.1	1.8	1.0	15.0	4.2	1.9	1.2	1.1	1.0
0.4	EWMA	20.9	8.7	5.5	4.0	3.2	2.0	11.9	4.4	2.7	2.1	1.7	1.0
	CNNSVR	20.2	7.1	3.0	2.0	1.4	1.0	13.4#	3.4	1.6	1.2	1.0	1.0
0.6	EWMA	194.1	81.1	30.1	15.9	10.7	5.5	852.2	443.5	56.0	16.3	9.3	4.2
	CNNSVR	171.0	66.3	24.9	10.2	5.9	2.3	634.3	343.1	50.8	12.8	6.1	1.7

show the out-of-control ARL values for shift sizes $\delta =0.9,0.8,0.7,0.6,0.5,\mathrm{and}0.2$, where # indicates when the CNNSVR chart is less efficient than the EWMA chart. Recall that the out-of-control ARL values of the CNNSVR chart can be obtained via the simulation procedure of Figure 3. For the case of $\lambda =0.1$ in Table 4, as the skewness values become small, the out-of-control ARL values for $Pc=0.2$ and 0.4 also decrease. Therefore, both charts with an approximately symmetric distribution exhibit better detection efficiency for the lower censored rate. In the case of $Pc=0.6$, as the skewness increases, both charts in all shift sizes become slow. Few out-of-control ARL values of the CNNSVR chart at a shift size of $\delta =0.9$ are larger than the values of the EWMA chart but, at other shift sizes, the CNNSVR chart outperforms the EWMA chart. The ARL values of both charts increase as the censoring rate increases. For most shift sizes, the CNNSVR chart demonstrates a superior ability to detect mean reduction.

As seen in Table 5, both charts produce the same results for $\lambda =0.1\mathrm{and}0.2$. However, when comparing Table 4 and Table 5, the detection efficiency of $\lambda =0.2$ outperforms that of $\lambda =0.1$ for most shift sizes. Thus, the choice of a larger $\lambda$ value significantly improves the detection efficiency.

For the cases of $\lambda =0.1\mathrm{and}0.2$, the mean reduction of $Pc=0.8$ cannot be effectively detected, because the ARL values of both charts at small shift sizes ($\delta =0.9,0.8,\mathrm{and}0.7$) almost exceed 1000. This study increases $\lambda$ to 0.8 for the censored data of $Pc=0.8$ to simulate the ARL values.

Table 6. ARL values of CNNSVR and EWMA charts for $Pc=0.8$.

The Parameters for $\mathbf{Weibull}\mathbf{Distribution}\mathbf{(}{\mathit{a}}_0,{\mathit{b}}_0\mathbf{)}$	Charts	$\mathbf{LCL}\mathbf{or}{\mathit{\eta }}^{\mathbf{*}}$	$$\mathbf{Shift}\mathbf{Size}\mathit{\delta }$$
			0.9	0.8	0.7	0.6	0.5	0.2
(1, 0.8)	EWMA	0.014	133.0	88.0	58.0	33.1	20.1	5.2
	CNNSVR	−0.016	92.4	86.6	56.4	31.8	28.4#	5.0
(1, 1)	EWMA	0.023	310.1	187.6	105.0	51.0	26.5	4.7
	CNNSVR	−0.030	107.6	74.7	67.6	50.5	35.2#	4.6
(1, 2.5)	EWMA	0.096	642.9	675.2	715.1	427.2	138.7	3.4
	CNNSVR	−0.109	112.3	83.3	92.4	118.4	101.7	3.2
(1, 4)	EWMA	0.136	5080.1	11,220.0	12,193.0	8056.0	3636.0	3.5
	CNNSVR	−0.078	317.9	1201.1	85.2	65.3	50.1	3.3

shows the ARL values, LCL, and ${\eta }^{\mathrm{\ast }}$ for the censored data of $Pc=0.8$. The CNNSVR still exhibits a better detection efficiency than the EWMA chart at most shift sizes for higher censored data. As the skewness values become smaller, CNNSVR is clearly better than EWMA statistic in terms of detection.

5. Illustrative Example for Implementing a CNNSVR-Based Residual Chart

For a liquid crystal display module (LCM), the reliability test is the primary method to determine its lifetime and it is conducted at a temperature of 70 °C and humidity of 80%. LCM lifetimes follow a Weibull distribution of $a_0=3.1$ and $b_0=2.2$. According to customer requirements, five units of each batch of LCMs must be randomly sampled to test lifetime values with a 60% censored rate (censored time is 2.284 h, and CEV is 0.997) before shipment. The process mean of an in-control state is 2.745. Practitioners referred to the parameters of the EWMA chart and developed the CNNSVR-based residual chart using $\lambda =0.2$ and the in-control ARL = 200.

Practitioners may refer to Section 3.1 to set up the CNNSVR-based residual chart, including generating 10,000 EWMA statistics for the training data, such that the training data consist of 9999 records.

Table 7. The generation of training data for LCM.

t	Simulation Data ${\mathit{s}}_{\mathit{i}}$					Conditional Expected Value ${\mathit{x}}_{\mathit{i}}$					Sample Mean $\overline{{\mathit{X}}_{\mathit{t}}}$	EWMA $\mathbf{Statistic}{\mathit{E}}_{\mathit{t}}$
1	0.734	1.581	2.115	1.665	1.851	0.734	0.997	0.997	0.997	0.997	0.944	2.385
2	0.839	0.936	2.312	1.983	1.659	0.839	0.936	0.997	0.997	0.997	0.953	2.099
3	0.936	0.660	1.846	2.532	2.615	0.936	0.660	0.997	0.997	0.997	0.917	1.863

shows the calculation of the first three EWMA statistics. The $s_i$ values are five random numbers generated by the Weibull distribution of $a_0=3.1$ and $b_0=2.2$. If $s_i>2.284$, then $x_i=0.997$; otherwise, $x_i=s_i$. Then, $\overline{X_t}$ is the average of the five $x_i$ values. The EWMA statistic $E_t$ is calculated using Equation (2). The first two records in this training dataset are (input, output) = $\left(2.385,2.099\right)$ and $\left(2.099,1.863\right)$. Following the procedure in Figure 2, the training dataset is inputted into the CNNSVR network and the optimal threshold value ${\eta }^{\mathrm{\ast }}$ is calculated to be $-0.124$.

Next, the CNNSVR-based residual chart is implemented to monitor the lifetime of LCM. Practitioners sample five units from each batch for life testing under the conditions of 70 °C and 80% humidity and halt testing when the test time reaches 1.568 h. Data from five testing units of each batch are used to calculate the EWMA statistic. The EWMA statistic is inputted into the CNNSVR network to predict the statistic of the next period. As shown in

Table 8. Predicted and error values of the CNNSVR network for the LCM lifetime test.

No. of Batch	Actual Value ${\mathit{E}}_{\mathit{t}}$	Predicted Value $\widehat{{\mathit{E}}_{\mathit{t}}}$	Error Value ${\mathit{E}}_{\mathit{t}}-\widehat{{\mathit{E}}_{\mathit{t}}}$	No. of Batch	Actual Value ${\mathit{E}}_{\mathit{t}}$	Predicted Value $\widehat{{\mathit{E}}_{\mathit{t}}}$	Error Value ${\mathit{E}}_{\mathit{t}}-\widehat{{\mathit{E}}_{\mathit{t}}}$
101	0.29	-	-	116	2.69	2.47	0.22
102	2.72	2.71	0.10	117	2.68	2.74	−0.06
103	2.69	2.58	0.11	118	2.67	2.49	0.18
104	2.74	2.03	0.71	119	2.75	2.74	0.01
105	0.83	0.84	−0.01	120	2.33	2.60	−0.27
106	1.36	1.35	0.01	121	1.72	2.04	−0.32
107	2.70	1.92	0.78	122	2.07	2.41	−0.34
108	2.75	2.87	−0.12	123	2.35	2.59	−0.24
109	2.75	2.63	0.12	124	0.12	0.45	−0.33
110	2.75	2.37	0.38	125	1.77	2.20	−0.43
111	2.72	2.24	0.48	126	1.85	2.41	−0.56
112	1.04	1.02	0.02	127	0.18	1.14	−0.96
113	1.01	1.08	−0.07	128	1.20	2.07	−0.87
114	0.06	0.05	0.01	129	1.04	2.10	−1.06
115	2.84	2.29	0.55	130	0.24	1.74	−1.50

, the EWMA statistic of the 101st batch is 0.29, so the practitioners input 0.29 into the CNNSVR network and the CNNSVR network outputs the predicted value of 2.71 for the 102nd batch. After the 102nd batch has been produced and the lifetime tests of five units are completed, the actual statistic of this batch is calculated to be 2.72 and the error value of this batch is 0.01 (=2.72 − 2.71). Practitioners plot the error value of 0.01 on the CNNSVR-based residual chart. Figure 5 shows the CNNSVR-based residual chart for the 29 error values shown in Table 8. The error values of batches 120–130 are below the ${\eta }^{\mathrm{\ast }}$ value, and so this chart signals a variation at the 120th batch.

6. Conclusions

In this paper, a residual chart combining a convolutional neural network (CNN) and support vector regression (SVR) with an exponentially weighted moving average (EWMA) and conditional expected value (CEV) statistics was proposed to detect the mean decrease for Weibull type-I right-censored data. As demonstrated by a comparison of the average run lengths, the CNNSVR-based residual chart generally outperforms the EWMA CEV chart at low, moderate, and high censoring rates. The CNNSVR-based residual chart for the Weibull parameters with approximate symmetry achieves better detection efficiency. As the skew coefficient value of the Weibull distribution increases, the detection efficiency decreases. In a heavily skewed Weibull distribution or higher censored rate, the use of a larger smoothing parameter of the EWMA statistic increases the performance of the CNNSVR-based residual chart. Future work could be extended to use double EWMA CEV statistics to monitor other types of censored data with normal or non-normal distributions. In addition, combining multiple deep-learning methods to build control charts is an interesting future study.