Cholesky Factorization Based Online Sequential Multiple Kernel Extreme Learning Machine Algorithm for a Cement Clinker Free Lime Content Prediction Model

Abstract

Aiming at the difficulty in real-time measuring and the long offline measurement cycle for the content of cement clinker free lime (fCaO), it is very important to build an online prediction model for fCaO content. In this work, on the basis of Cholesky factorization, the online sequential multiple kernel extreme learning machine algorithm (COS-MKELM) is proposed. The LDL${}^T$ form Cholesky factorization of the matrix is introduced to avoid the large operation amount of inverse matrix calculation. In addition, the stored initial information is utilized to realize online model identification. Then, three regression datasets are used to test the performance of the COS-MKELM algorithm. Finally, an online prediction model for fCaO content is built based on COS-MKELM. Experimental results demonstrate that the fCaO content model improves the performance in terms of learning efficiency, regression accuracy, and generalization ability. In addition, the online prediction model can be corrected in real-time when the production conditions of cement clinker change.

1. Introduction

Cement is an important building material. Its quality will affect the building safety directly [1,2]. In addition, the cement clinker is a key product during cement production. At the same time, the content of cement clinker free lime (fCaO) is an important index to evaluate the clinker quality [3]. It is the main factor that impacts the cement stability and clinker strength. Thus, operators and engineers should adjust the variables of cement production according to the fCaO content in real time. In the process of cement production, the cement clinker fCaO content should be controlled within a certain range [4]. When the fCaO content is higher than the standard range, the cement clinker is unqualified. On the other hand, when it is lower than the standard range, the clinker is over-burning and the energy consumption increases [5].

At present, common measuring methods of fCaO content are chemical analysis and fluorescence analyzers. The measurement of cement clinker fCaO content is mainly obtained by manual sampling intermittently, and then the sample cement clinker are sent to the laboratory for manual analysis [6]. According to the requirements of different cement production lines, the time interval of laboratory testing fCaO content varies from 1 to 4 h. However, there is a certain time difference between each process in the cement clinker firing process, which makes the measurement results of fCaO content have a great lag for the production guidance in the clinker firing process. The measurement results can not meet the requirements of real-time optimization control in the cement clinker firing process [7]. Therefore, the essential work of estimating fCaO content is to set up a high precision online prediction model which is very important for monitoring and optimizing the operations of cement plants.

Considering the nonlinearity and uncertainty between the fCaO content and the production process related variables, some scholars have established the regression prediction model by using the method of neural networks modeling. Fuzzy entropy and a neural-net ensemble with random weights are used to build the soft sensor of fCaO content [8]. Szatvanyi et al. [9] and Li et al. [10] combined flame image features and process variables methods to build the prediction model for fCaO content. Liu et al. [11] proposed a support vector machine ensemble model to estimate fCaO content. A time series analysis method [12] and an improved combination modeling method [13] were used for soft measurement of cement clinker fCaO content. Some neural network methods of establishing a prediction model in the cement calcination process were also proposed, such as time-varying delay deep belief network [14], multi-channel CNN with moving window [15], deep belief network with sliding window [16], and two-dimensional convolutional neural network [17]. In addition, other researchers working on the online prediction model for fCaO content have been published. Pani et al. [18,19] used a feed-forward artificial neural network and fuzzy inference to build a soft sensor model for online prediction fCaO content. The multivariate time series analysis and convolutional neural network method [20] and the kernel extreme learning machine algorithm [21] were proposed for online cement clinker quality monitoring.

Some of the above scholars established the prediction models of cement clinker fCaO content based on the offline neural network. However, when the training sample data is too large or the working conditions of cement clinker firing changed, the modeling process of the prediction model needs to be repeated. This will greatly increase the computational amount of the cement clinker fCaO content modeling process. In addition, some of the above scholars established the prediction models of cement clinker fCaO content based on the online neural network. However, due to the complexity of the cement clinker firing process, the performance of the online prediction model of the clinker fCaO content is poor, and the learning speed of parameters training is slow, which leads to the poor prediction effect of the cement clinker fCaO content. Therefore, it is necessary to deeply study the new online sequential learning neural network algorithm applicable to the cement clinker fCaO content prediction model. This can not only solve the problem of establishing and predicting the cement clinker fCaO content prediction model but also overcome the disadvantages of the existing conventional neural network algorithm, such as poor network performance and slow learning speed.

In recent years, the extreme learning machine (ELM) [22,23] and kernel extreme learning machine (KELM) [24] are proposed and have been successfully applied to model identification, such as saliency detection [25], gesture recognition [26], image classification [27], nonlinear fault detection [28], seepage time soft sensor model of nonwoven fabric [29], and rolling bearing sub-health recognition [30]. However, different kernel functions have different characteristics, and the performances are varying in different applications. Thus, the multiple kernel extreme learning machine (MKELM) has been proposed in [31] as a combination of multiple kernel learning and ELM, and it has also been successfully applied to many applications, such as network intrusion detection [32,33] and human activity recognition [34]. Considering a strong nonlinear relationship between the fCaO content and the related variables during cement production, MKELM is adopted to build an online prediction model for fCaO content in this paper. However, it is necessary to study how to select the appropriate kernel functions for the cement clinker fCaO content online prediction model.

In addition, Liang et al. [35] developed the online sequential extreme learning machine (OS-ELM) algorithm, and the online sequential kernel extreme learning machine (OS-KELM) algorithm is proposed by Wang and Han [36]. These online learning algorithms have received widespread attention in relevant research, such as modeling time series [37], dynamic modeling [38], batch process [39], and discharge forecasting [40]. On account of the large calculation of inverse matrix and the computational complexity of online sequential algorithm, the Cholesky factorization method is introduced to the extreme learning process [41,42]. However, the OS-ELM and OS-KELM algorithms need to compute the inverse of corresponding matrixes in each online sequential learning process. When the added data are too large, the matrix dimension in the network learning process of OS-ELM algorithm and OS-KELm algorithm is too large, which increases the computation of online sequential learning process and affects the learning speed of online training [43]. Therefore, in view of the limitations of the existing ELM algorithm, it is necessary to further study the neural network algorithm for online sequence learning to improve the structure of the existing neural network online learning.

In this research, considering different kernel function characteristics, three typical kernel functions are selected to construct an equivalent kernel function which is used in the MKELM algorithm. In order to avoid the large calculation of the inverse matrix and reduce the computational complexity of online sequential learning process, the LDL${}^T$ form Cholesky factorization based online sequential multiple kernel extreme learning machine (COS-MKELM) algorithm is proposed. In the COS-MKELM algorithm, there are two major differences: the LDL${}^T$ form Cholesky factorization of matrix is introduced, and the new matrix elements are calculated recursively based on the original matrix stored information. Then, three classical UCI regression datasets are used, which are available at https://www.dcc.fc.up.pt/~ltorgo/Regression/DataSets.html, on 5 December 2020, to test the validity of COS-MKELM. In addition, the cement clinker fCaO content online prediction model is built by the COS-MKELM algorithm. Simulation results show that the online prediction model has good learning efficiency, regression accuracy, and generalization ability.

The rest of this paper is arranged as follows. In Section 2, the MKELM algorithm with three typical kernel functions is derived. The COS-MKELM algorithm is proposed and tested in Section 3. In Section 4, the cement clinker fCaO online prediction model is built and simulation verified. Finally, Section 5 concludes the paper.

2. The MKELM Algorithm

In this section, the MKELM algorithm is given to handle different heterogeneous data integrations. It can improve the model precision compared with single kernel function. The optimal kernel of MKELM is expressed as a linear combination of multiple different basic kernel functions.

Assuming the set of training data are ${\mathit{x}}_i,{\mathit{y}}_i$, where $i=1,2,\cdots ,N$, N is the number of samples. ${\mathit{x}}_i={[x_{i1},x_{i2},\cdots ,x_{in}]}^T\in {\mathit{R}}^n$ denotes the input vector and ${\mathit{y}}_i={[y_{i1},y_{i2},\cdots ,y_{im}]}^T\in {\mathit{R}}^m$ is the output vector. n expresses the number of input layer nodes and m represents the number of output layer nodes. Then, the output function of MKELM can be given as follows:

$$\begin{array}{c}{f}_L\left({\mathit{x}}_i\right)={\displaystyle \sum _{k=1}^Q}{\displaystyle \sum _{p=1}^G}{\lambda }_p{\mathit{\beta }}_{kp}{g}_p\left({\mathit{\omega }}_k{\mathit{x}}_i+{\mathit{b}}_k\right)={\displaystyle \sum _{k=1}^Q}{\displaystyle \sum _{p=1}^G}{\lambda }_p{\mathit{\beta }}_{kp}{h}_p\left({\mathit{x}}_i\right)\end{array}$$

(1)

where ${\lambda }_p⩾0$ is a coefficient of the pth kernel function, $p=1,2,\cdots ,G$, G represents the number of base kernel functions; Q is the node number of hidden layer; ${\mathit{\omega }}_k={[{\omega }_{k1},{\omega }_{k2},\cdots ,{\omega }_{kl},\cdots ,{\omega }_{kn}]}^T$ is the weight of connection between the kth node of hidden layer and each input node, $l=1,2,\cdots ,n$; The ${\mathit{b}}_k$ sets threshold of the kth hidden node; ${\mathit{\beta }}_k={[{\beta }_{k1},{\beta }_{k2},\cdots ,{\beta }_{kq},\cdots ,{\beta }_{km}]}^T$ is the weight vector between hidden nodes and output nodes, $q=1,2,\cdots ,m$; $g\left(x\right)$ is the activation function.

According to the structural risk minimization principle, Equation (1) can be written as the minimum optimization problem:

$$\begin{array}{c}\underset{\lambda }{min}\underset{\beta ,\xi }{min}\frac{1}{2}{\displaystyle \sum _{p=1}^G}{\lambda }_p∥{\mathit{\beta }}_p∥{}^2+\frac{1}{2}\gamma {\displaystyle \sum _{i=1}^N}{\mathit{\xi }}_i^2\hfill \\ s.t.\left\{\begin{array}{c}{\displaystyle \sum _{p=1}^G}{h}_p\left({\mathit{x}}_i\right){\mathit{\beta }}_p={\mathit{y}}_i-{\mathit{\xi }}_i,i=1,2,\cdots ,N\hfill \\ {\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}=1,p=1,2,\cdots G\hfill \end{array}\right.\hfill \end{array}$$

(2)

where $\gamma$ is regularization coefficient, and ${\mathit{\xi }}_i$ is the error between the ith actual output and expected output.

It is not difficult to verify that (2) is a joint-convex optimization problem, and the solution of (2) with Lagrangian method can be written as:

$$\begin{array}{c}L(\mathit{\beta },\mathit{\xi },\mathit{\alpha },\tau )=\frac{1}{2}{\displaystyle \sum _{p=1}^G}{\lambda }_p∥{\mathit{\beta }}_p∥{}^2+\frac{\gamma }{2}{\displaystyle \sum _{i=1}^N}{\mathit{\xi }}_i^2\hfill \\ -{\displaystyle \sum _{i=1}^N}{\mathit{\alpha }}_i\{{\displaystyle \sum _{p=1}^G}{h}_p\left({\mathit{x}}_i\right){\mathit{\beta }}_p+{\mathit{\xi }}_i-{\mathit{y}}_i\}-{\displaystyle \sum _{i=1}^N}{\tau }_i\{{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}-1\}\hfill \end{array}$$

(3)

where $\mathit{\alpha }$ and $\tau$ are the Lagrange multipliers.

According to the Karush–Kuhn–Tucker optimality condition, the partial differential of the Lagrangian function in (3) with respect to the variables ($\mathit{\beta }$, $\mathit{\xi }$, $\mathit{\alpha }$ and $\tau$) can be calculated respectively as:

$$\left\{\begin{array}{c}\frac{\partial L}{\partial \mathit{\beta }}={\lambda }_p{\mathit{\beta }}_p-{\displaystyle \sum _{i=1}^N}{\alpha }_i{h}_p\left({\mathit{x}}_i\right)=0\hfill \\ \frac{\partial L}{\partial \mathit{\xi }}=\gamma {\mathit{\xi }}_i-{\mathit{\alpha }}_i=0\hfill \\ \frac{\partial L}{\partial \mathit{\alpha }}={\displaystyle \sum _{p=1}^G}{h}_p\left({\mathit{x}}_i\right){\mathit{\beta }}_p+{\mathit{\xi }}_i-{\mathit{y}}_i=0\hfill \\ \frac{\partial L}{\partial \tau }={\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}-1=0\hfill \end{array}\right.$$

(4)

From the above formula, the following formula can be obtained:

$$\left\{\begin{array}{c}{\mathit{\beta }}_p=\frac{1}{{\lambda }_p}{\displaystyle \sum _{i=1}^N}{\alpha }_i{h}_p\left({\mathit{x}}_i\right)\hfill \\ {\mathit{\xi }}_i=\frac{{\mathit{\alpha }}_i}{\gamma }\hfill \\ {\mathit{y}}_i={\displaystyle \sum _{p=1}^G}{h}_p\left({\mathit{x}}_i\right){\mathit{\beta }}_p+{\mathit{\xi }}_i\hfill \\ {\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}=1\hfill \end{array}\right.$$

(5)

Therefore, the Lagrange multiplier $\tau$ vanishes during the derivation process. Then, let $\mathit{\beta }$ and $\mathit{\xi }$ also vanish according to (5), and the following formula can be obtained:

$$f_L\left({\mathit{x}}_i\right)={\mathit{y}}_i={\displaystyle \sum _{p=1}^G}{\displaystyle \sum _{i=1}^N}\frac{{\alpha }_i}{{\lambda }_p}{h}_p^T\left({\mathit{x}}_i\right)h_p\left({\mathit{x}}_i\right)+\frac{{\alpha }_i}{\gamma }$$

(6)

According to (4) and (5), the (6) can be rewritten into a matrix form as:

$$(\frac{\mathit{I}}{\gamma }+\mathsf{\Omega })\mathit{\alpha }=\mathit{Y}$$

(7)

where $\mathsf{\Omega }$ is the multiple kernel function matrix of MKELM, whose expression can be written as:

$$\mathsf{\Omega }={\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{h}_p^T\left({\mathit{x}}_i\right)h_p\left({\mathit{x}}_i\right)={\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}K(\mathit{x},{\mathit{x}}_i)$$

(8)

It is the multiple kernel functions matrix of the MKELM. From (7), the $\mathit{\alpha }$ corresponding to the structural parameter of MKELM can be obtained by:

$$\mathit{\alpha }={\left(\frac{\mathit{I}}{\gamma }+\mathsf{\Omega }\right)}^{-1}\mathit{Y}$$

(9)

Considering the general characteristics of kernel functions and the model complexity, a new equivalent kernel is constructed, which uses the weighted combination of Polynomial kernel, Exponential Radial Basis kernel, and the Gaussian Radial Basis kernel:

Polynomial kernel:

$${\mathit{K}}_{\mathrm{poly}}={[\left({x_i}^{T}x\right)+C]}^3,C>0$$

(10)

Exponential Radial Basis kernel:

$${\mathit{K}}_{exp}=exp\left(-∥x-x_i∥/\left(2{{\sigma }^{\prime }}^2\right)\right)$$

(11)

Gaussian Radial Basis kernel:

$${\mathit{K}}_{\mathrm{RBF}}=exp(-\left({∥x-x_i∥}^2\right)/\left(2{\sigma }^2\right))$$

(12)

where C, ${\sigma }^{\prime }$ and $\sigma$ are parameters to be determined.

The equivalent kernel function of MKELM can be written as:

$$\mathsf{\Omega }=\frac{1}{{\lambda }_1}{\mathit{K}}_{\mathrm{poly}}+\frac{1}{{\lambda }_2}{\mathit{K}}_{exp}+\frac{1}{{\lambda }_3}{\mathit{K}}_{\mathrm{RBF}}$$

(13)

where ${\lambda }_1$, ${\lambda }_2$, and ${\lambda }_3$ are the coefficients of corresponding kernel functions.

According to the constraint conditions of (2) and letting the coefficient ${\lambda }_3$ in (13) vanish, Equation (13) can be simplified as:

$${\mathsf{\Omega }}_{MKELM}=\frac{1}{{\lambda }_1}{\mathit{K}}_{\mathrm{poly}}+\frac{1}{{\lambda }_2}{\mathit{K}}_{exp}+(1-\frac{1}{{\lambda }_1}-\frac{1}{{\lambda }_2}){\mathit{K}}_{\mathrm{RBF}}$$

(14)

Therefore, the expression of the MKELM depicted in this research can be written as:

$$\begin{array}{c}{f}_L\left({\mathit{x}}_i\right)={\displaystyle \sum _{i=1}^N}{\alpha }_i\{\frac{1}{{\lambda }_1}·{[\left({{\mathit{x}}_i}^T\mathit{x}\right)+C]}^3+\frac{1}{{\lambda }_2}·exp(-\frac{∥\mathit{x}-{\mathit{x}}_i∥}{2{{\sigma }^{\prime }}^2})\hfill \\ +(1-\frac{1}{{\lambda }_1}-\frac{1}{{\lambda }_2})·exp(-\frac{{∥\mathit{x}-{\mathit{x}}_i∥}^2}{2{\sigma }^2})+\frac{1}{\gamma }\}\hfill \end{array}$$

(15)

In summary, the parameters $\gamma$, C, ${\sigma }^{\prime }$, $\sigma$, and $\lambda$ in (15) can be determined empirically, and the parameter $\mathit{\alpha }$ is obtained by matrix inversion calculation according to (9). Therefore, the main amount of calculation in MKELM algorithm is the solution of parameter $\mathit{\alpha }$.

3. The Proposed COS-MKELM Algorithm and Performance Verification

It can be seen from the MKELM algorithm depicted in Section 2 that, if the modeling process is repeated with the new input training data, the parameter $\mathit{\alpha }$ of MKELM needs to be obtained by repeating the computation of the inverse matrix. In addition, when the new training data added into MKELM are too large, calculating the matrix inverse with a very large dimension will greatly increase the amount of calculation in MKELM online modeling. Thus, the ${LDL}^T$ form Cholesky factorization method is used in this research to avoid the large matrix inverse calculation and simplify the online sequential calculation of MKELM. In this section, the COS-MKELM algorithm is proposed.

3.1. The Solution of COS-MKELM Parameter by Cholesky Factorization

According to (8) and (9), the expression of parameter $\mathit{\alpha }$ can be simplified as (16), where N is the number of initial training samples:

$$\begin{array}{c}\mathit{\alpha }={\left({\mathit{I}}_N/\gamma +{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{\mathit{H}}_{p,N}{\mathit{H}}_{p,N}^T\right)}^{-1}{\mathit{Y}}_N\end{array}$$

(16)

Denote ${\mathit{E}}_N={\mathit{I}}_N/\gamma +{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{\mathit{H}}_{p,N}{\mathit{H}}_{p,N}^T$, and it can be obtained as ${\mathit{E}}_N^T={\mathit{E}}_N$. Then, the (16) can be transformed as (17) when left multiplied matrix ${\mathit{E}}_N$:

$$\begin{array}{c}{\mathit{E}}_N\mathit{\alpha }={\mathit{Y}}_N\end{array}$$

(17)

The Cholesky factorization method can be used to solve the linear equation shown in (17). However, the conventional Cholesky factorization needs to be a square root, which is easy to reduce the computational precision and increase the computational complexity. In order to overcome this shortage, the matrix ${\mathit{E}}_N$ is factorized as the ${LDL}^T$ form Cholesky factorization. In addition, the elements in the ${LDL}^T$ forming the Cholesky factorization matrix are calculated recursively. Before getting the ${LDL}^T$ form Cholesky factorization, it is necessary to prove that the matrix ${\mathit{E}}_N$ is a symmetric positive definite matrix. Assume that any column vector ${\mathit{W}}_N={\left[\begin{array}{cccc}{W}_1\hfill & W_2\hfill & \cdots \hfill & W_N\hfill \end{array}\right]}^T\ne \mathbf{0}$ has the same dimension as ${\mathit{E}}_N$, and the following formula can be obtained:

$$\begin{array}{c}{\mathit{W}}_N^T{\mathit{E}}_N{\mathit{W}}_N={\mathit{W}}_N^T\left({\mathit{I}}_N/\gamma +{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{\mathit{H}}_{p,N}{\mathit{H}}_{p,N}^T\right){\mathit{W}}_N\hfill \\ =\frac{1}{\gamma }{\mathit{W}}_N^T{\mathit{W}}_N+{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{\mathit{W}}_N^T{\mathit{H}}_{p,N}{\mathit{H}}_{p,N}^T{\mathit{W}}_N\hfill \\ =\frac{1}{\gamma }\left[{\mathit{W}}_N,{\mathit{W}}_N\right]+{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}\left[{\mathit{H}}_{p,N}^T{\mathit{W}}_N,{\mathit{H}}_{p,N}^T{\mathit{W}}_N\right]\hfill \\ =\frac{1}{\gamma }{∥{\mathit{W}}_N∥}^2+{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{∥{\mathit{H}}_{p,N}^T{\mathit{W}}_N∥}^2>0\hfill \end{array}$$

(18)

Equation (18) proves that the matrix ${\mathit{E}}_N$ is a symmetric positive definite matrix. Then, ${\mathit{E}}_N$ can be factorized to ${LDL}^T$ form as follows:

$$\begin{array}{c}{\mathit{E}}_N={\mathit{L}}_N{\mathit{D}}_N{\mathit{L}}_N^T\end{array}$$

(19)

where ${\mathit{L}}_N$ is a unit lower triangular matrix and ${\mathit{D}}_N$ is a diagonal matrix with diagonal positive.

From (19), (20) can be obtained:

$$\begin{array}{c}{e}_{ij}={\displaystyle \sum _{k=1}^{j-1}}{l}_{ik}{d}_k{l}_{jk}+l_{ij}{d}_j\end{array}$$

(20)

where e, l, and d are the elements of matrix ${\mathit{E}}_N$, ${\mathit{L}}_N$ and ${\mathit{D}}_N$, $1\le j\le i\le N$.

Thus, the elements of ${\mathit{L}}_N$ and ${\mathit{D}}_N$ can be calculated recursively as follows:

$$\begin{array}{c}\left\{\begin{array}{c}{d}_j=e_{jj}-{\displaystyle \sum _{k=1}^{j-1}}{l}_{jk}^2{d}_k\hfill \\ l_{ij}=\left(e_{ij}-{\displaystyle \sum _{k=1}^{j-1}}{l}_{ij}{l}_{jk}{d}_k\right)/d_j\hfill \end{array}\right.\end{array}$$

(21)

where $k=1,2,\dots ,j-1$; $i=j+1,j+2,\dots ,N$; $j=1,2,\dots ,N$.

Put (19) into (17) and multiply ${\mathit{L}}_N^{-1}$ on both sides of (17):

$$\begin{array}{c}{\mathit{D}}_N{\mathit{L}}_N^T\mathit{\alpha }={\mathit{L}}_N^{-1}{\mathit{Y}}_N={\mathit{M}}_N\end{array}$$

(22)

where ${\mathit{M}}_N$ is a column vector.

The elements $m_i$ in matrix ${\mathit{M}}_N$ can be obtained as follows:

$$m_i=\left\{\begin{array}{cc}{y}_i/l_{ii}\hfill & i=1\hfill \\ \left(y_i-{\displaystyle \sum _{j=1}^{i-1}}{l}_{ij}{m}_j\right)/l_{ii}\hfill & i>1\hfill \end{array}\right.$$

(23)

where $i=1,2,\dots ,N$.

According to (22), the elements ${\alpha }_i$ in the parameter $\mathit{\alpha }$ can be calculated recursively as follows:

$${\alpha }_i=\left\{\begin{array}{cc}{m}_i/d_i\hfill & i=N\hfill \\ m_i/d_i-{\displaystyle \sum _{j=i+1}^N}{l}_{ij}{\alpha }_j\hfill & i<N\hfill \end{array}\right.$$

(24)

Therefore, in the solution of the COS-MKELM parameter calculation process, the ${LDL}^T$ form Cholesky factorization method avoids the large matrix inverse and extracting a root calculation.

3.2. Online Sequential Learning COS-MKELM Parameter

During the online sequential learning process of the COS-MKELM algorithm, the new training data, which were added into the machine, will be learned immediately. Assuming that the number of new training data is $\widehat{N}$, then the matrix ${\mathit{E}}_{N+\widehat{N}}$ can be written as follows:

$$\begin{array}{c}{\mathit{E}}_{N+\widehat{N}}={\mathit{I}}_{N+\widehat{N}}/\gamma +{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{\mathit{H}}_{p,N+\widehat{N}}{\mathit{H}}_{p,N+\widehat{N}}^T\hfill \\ ={\mathit{I}}_{N+\widehat{N}}/\gamma +{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}\left[\begin{array}{c}{\mathit{H}}_{p,N}\\ {\mathit{H}}_{p,\widehat{N}}\end{array}\right]{\left[\begin{array}{c}{\mathit{H}}_{p,N}\hfill \\ {\mathit{H}}_{p,\widehat{N}}\hfill \end{array}\right]}^T\hfill \end{array}$$

(25)

According to ${\mathit{E}}_N$ described in Section 3.1, (25) can be expanded as follows:

$$\begin{array}{c}{\mathit{E}}_{N+\widehat{N}}=\left[\begin{array}{cc}{\mathit{E}}_N& {\mathit{E}}_N^1\\ {\mathit{E}}_N^2& {\mathit{E}}_N^3\end{array}\right]\hfill \\ {\mathit{E}}_N^1={\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}\left(\begin{array}{c}{\mathit{H}}_{p,1}{\mathit{H}}_{p,\widehat{N}}^T\\ ⋮\\ {\mathit{H}}_{p,N}{\mathit{H}}_{p,\widehat{N}}^T\end{array}\right)\hfill \\ {\mathit{E}}_N^2={\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}\left({\mathit{H}}_{p,\widehat{N}}{\mathit{H}}_{p,1}^T,\cdots ,{\mathit{H}}_{p,\widehat{N}}{\mathit{H}}_{p,N}^T\right)\hfill \\ {\mathit{E}}_N^3={\mathit{I}}_{N+\widehat{N}}/\gamma +{\displaystyle \sum _{p=1}^G}\frac{1}{{\lambda }_p}{\mathit{H}}_{p,\widehat{N}}{\mathit{H}}_{p,\widehat{N}}^T\hfill \end{array}$$

(26)

The matrix ${\mathit{E}}_{N+\widehat{N}}$ can be factorized in the ${LDL}^T$ form as follows:

$$\begin{array}{c}{\mathit{E}}_{N+\widehat{N}}={\mathit{L}}_{N+\widehat{N}}{\mathit{D}}_{N+\widehat{N}}{\mathit{L}}_{N+\widehat{N}}^T\end{array}$$

(27)

It can be obtained from (19) as follows:

$$\begin{array}{c}{\widehat{e}}_{ij}={\displaystyle \sum _{k=1}^{j-1}}{\widehat{l}}_{ik}{\widehat{d}}_k{\widehat{l}}_{jk}+{\widehat{l}}_{ij}{\widehat{d}}_j\end{array}$$

(28)

Here, $\widehat{e}$, $\widehat{l}$, and $\widehat{d}$ are the elements of matrix ${\mathit{E}}_{N+\widehat{N}}$,${\mathit{L}}_{N+\widehat{N}}$ and ${\mathit{D}}_{N+\widehat{N}}$, $1\le j\le i\le N+\widehat{N}$.

Form (26) and (27), it can be seen that, when $1\le j\le i\le N$, ${\mathit{L}}_{N+\widehat{N}}$ and ${\mathit{D}}_{N+\widehat{N}}$ are the same to ${\mathit{L}}_N$ and ${\mathit{D}}_N$. Only the remaining new training data (the $\widehat{N}$ new training data) need to be calculated recursively according to (21). Thus, the elements of matrix ${\mathit{L}}_{N+\widehat{N}}$ and ${\mathit{D}}_{N+\widehat{N}}$ can be calculated recursively based on matrix ${\mathit{L}}_N$ and ${\mathit{D}}_N$.

Similarly, the matrix ${\mathit{M}}_{N+\widehat{N}}$ can be obtained according to (23). It is shown as (29):

$${\mathit{M}}_{N+\widehat{N}}=\left[\begin{array}{c}{\mathit{M}}_N\hfill \\ m_{\widehat{N}}\hfill \end{array}\right]$$

(29)

The elements ${\mathit{\alpha }}_i$ in the new parameter $\mathit{\alpha }$ of COS-MKELM can be obtained according to (24). Substituting the parameter $\mathit{\alpha }$ into (15), the output of COS-MKELM algorithm can be obtained. In summary, when the new training data are added into COS-MKELM, the new matrix elements are calculated recursively based on the original matrix information. Thus, the COS-MKELM algorithm has reduced the computational complexity and simplified the online sequential learning process.

3.3. COS-MKELM Performance Verification

In this subsection, the regression datasets of Auto MPG ($d_1$), Machine CPU ($d_2$), and Boston Housing ($d_3$) are used to test the performance of COS-MKELM. These regression datasets have been widely used to test the performance of the modeling method. All of the experiments are carried out under Windows 7 and Matlab R2010a, with 4 GB RAM, and an Inter Pentium G2020 CPU with 2.90 GHz.

The COS-MKELM algorithm is compared with the OS-ELM algorithm and the OS-KELM algorithm. The activation function of OS-ELM algorithm is Sigmoid function, and the number of hidden layer nodes of OS-ELM algorithm is 40. The kernel function of OS-KELM is selected as the Gaussian Radial Basis kernel, and the kernel function of the proposed COS-MKELM algorithm is the same as the kernel function presented in Section 2. The parameters of COS-MKELM algorithm are set as the following: C = 60, ${\sigma }^{\prime }$ = 2.5, $\sigma$ = 8, ${\lambda }_1$ = 10, ${\lambda }_2$ = 2.5, $\gamma$ = 10,000. 3/4 of each regression datasets is used as training data, and the remaining 1/4 is used as testing data. The number of initial training data of OS-ELM algorithm is set as 50, and the rest of the training data are added in 10%. In addition, 10% of the training data are added in OS-KELM and COS-MKELM each time.

In this research, the root mean square error (RMSE) is used to evaluate the precision performance index. In addition, RMSE can be expressed as follows:

$$\begin{array}{c}RMSE=\sqrt{\frac{1}{N_T}{\displaystyle \sum _{i=1}^{N_T}}{\left(y_i-{\widehat{y}}_i\right)}^2}\end{array}$$

(30)

where $y_i$ and ${\widehat{y}}_i$ stand for the actual value and estimated value, and $N_T$ is the number of testing data.

The training time and RMSE for training data are listed in

Table 1. Performance comparison of the three learning algorithms on UCI regression datasets.

Dataset		OS-ELM	OS-KELM	COS-MKELM
$d_1$	Training time/s	0.8217	1.0545	0.7145
	RMSE	2.54	2.19	1.73
$d_2$	Training time/s	0.6529	0.9015	0.5648
	RMSE	23.15	21.17	16.58
$d_3$	Training time/s	0.9753	1.2045	0.8210
	RMSE	3.82	3.65	2.97

. In addition, the output values of each regression datasets for testing data are shown in Figure 1, Figure 2 and Figure 3.

From Table 1, it can be seen that, in the COS-MKELM algorithm, the training times of the three regression datasets are all the smallest. It demonstrates that the COS-MKELM algorithm is better than the other comparison algorithms in terms of learning efficiency. As seen from the RMSE values for testing data listed in Table 1 and the output results shown in Figure 1, Figure 2 and Figure 3, the predicted testing values based on COS-MKELM are closer to the actual values than the other comparison algorithms. It is concluded that the COS-MKELM algorithm proposed in this research shows much better performance in regression accuracy and generalization ability than the OS-ELM and OS-KELM.

Therefore, it can be seen that it is feasible to build a cement clinker fCaO online prediction model based on the COS-MKELM algorithm proposed in this paper.

4. Cement Clinker fCaO Model and Simulation

4.1. fCaO Content Online Prediction Model

In this subsection, the proposed COS-MKELM algorithm is adopted to build the cement clinker fCaO content online prediction model. The flowchart of the building cement clinker fCaO content online prediction model is shown in Figure 4.

Combined with the COS-MKELM algorithm, the fCaO online prediction modeling steps can be described in detail below:

• Step 1: Collect the dataset of variables related to the fCaO content from the cement production line. Given the initial training data ${\left\{{\mathit{x}}_i,{\mathit{y}}_i\right\}}_{i=1}^N$. Initialize the values of $\gamma$, C, ${\sigma }^{\prime }$, $\sigma$, ${\lambda }_1$ and ${\lambda }_2$.
• Step 2: Factorize the matrix ${\mathit{E}}_N$ to ${LDL}^T$ form. Calculate the elements of ${\mathit{L}}_N$, ${\mathit{D}}_N$, and ${\mathit{M}}_N$ recursively according to (21) and (23).
• Step 3: Calculate the parameter $\mathit{\alpha }$ of COS-MKELM according to (24), and put it into (15) to obtain the online prediction output value of fCaO content.
• Step 4: If the new training data are added into the COS-MKELM, update the number of online learning steps $k=k+1$ and run Step 5 to learn the new data; otherwise, output the parameter values ($\mathit{\alpha }$, $\gamma$, C, ${\sigma }^{\prime }$, $\sigma$, ${\lambda }_1$ and ${\lambda }_2$), and then obtain the fCaO content online prediction model.
• Step 5: Assume the new training data added into COS-MKELM is ${\left\{{\mathit{x}}_i,{\mathit{y}}_i\right\}}_{i=1}^{\widehat{N}}$. Based on the matrix ${\mathit{L}}_k$, ${\mathit{D}}_k$ and ${\mathit{M}}_k$ obtained from the kth step of online sequential learning, calculate the elements of matrix ${\mathit{L}}_k+1$, ${\mathit{D}}_k+1$ and ${\mathit{M}}_k+1$ recursively according to (21) and (23). Return Step 3.

After the cement clinker fCao content online prediction model is obtained, in the process of fCao content online prediction, the parameters of fCaO model are updated constantly. When new sample data are obtained or the production conditions of cement clinker change, the above modeling Step 4 is repeated on the basis of the existing online prediction model. The cement clinker fCaO content online prediction model is trained continuously, and relevant parameters of the model are updated. The online process of updating parameters makes the model have strong applicability and anti-interference ability.

4.2. Dataset and Ascertaining Model Parameters

In this research, the raw dataset includes simultaneous process variables and fCaO content measured values under various conditions for fCaO online prediction modeling. It is obtained from No.1 rotary kiln at the TangshanJidong cement plant with 5000 tons clinker production capacity per day. Through object linking and embedding for process control communication technology, the input variables data of cement clinker fCaO content prediction model were collected from the distributed control system of the cement clinker production line. The laboratory samples the clinker on site every hour for chemical testing to obtain the actual value of the cement clinker fCaO content, which are the output variable data of the cement clinker fCaO content prediction model. Thus, the data of cement clinker fCaO content prediction model are sampled once every hour, with 24 samples per day.

Because the collected data are affected by uncertain factors such as noise, the continuous mean filtering method is used to pre-process the collected initial data. After filtering the collected initial data, a total of 215 pairs of data are taken as the training and testing data sets of the cement clinker fCaO content prediction model. From these data, 150 data are selected randomly as the training dataset (used for fCaO content online prediction model development), and the other 65 data are testing datasets (used for fCaO content online prediction model validation). Some of the collected process variables are shown in

Table 2. Some collected process variables data samples.

Variables	1	2	3	…	214	215
$W_r$/(t/h)	390	390	388	…	380	387
$W_{cc}$/(t/h)	7.4	7.8	8.3	…	7.7	8.6
$W_{kc}$/(t/h)	11.9	12.1	12.8	…	13.1	12.7
$S_f$/(r/min)	815	824	829	…	837	841
$T_c$/${}^{°}$C	890	894	892	…	897	886
$T_{kt}$/${}^{°}$C	997	993	1002	…	1007	992
$T_{kh}$/${}^{°}$C	1018	1023	1014	…	1035	1024
$C_n$/ppm	868	878	984	…	682	719
$I_k$/A	886	884	849	…	812	919
$KH$/%	0.94	0.95	0.92	…	0.99	0.98
$SM$/%	2.71	2.74	2.73	…	2.74	2.76
$AM$/%	1.10	1.16	1.17	…	1.37	1.35
fCaO/%	1.33	0.78	0.31	…	0.63	0.90

$W_r$ = weight of raw material; $W_{cc}$ = weight of calciner coal; $W_{kc}$ = weight of kiln coal; $S_f$ = induced draft fan speed; $T_c$ = temperature of calciner exit gas; $T_{kt}$ = temperature of kiln tail; $T_{kh}$ = temperature of kiln head; $C_n$ = content of smoke chamber NOx; $I_k$ = kiln main motor current; $KH$ = lime saturation factor; $SM$ = silica modulus; $AM$ = alumina-oxygen modulus. The fCaO content would be taken as the output of online prediction model, and the above 12 variables are chosen as the inputs of online prediction model.

.

In order to prove the validity of the fCaO online prediction model based on COS-MKELM, OS-ELM and OS-KELM are used as comparative algorithms in this research. The parameters of the three models are set to the same as the parameters in Section 3.3.

4.3. Results and Discussion

In this subsection, the RMSE, mean absolute percentage error (MAPE), mean absolute error (MAE), maximum absolute error (MAXE), and coefficient of determination ($R^2$) are used as performance assessment indexes for the fCaO models’ prediction. The expressions of RMSE are the same as (30), and the other criteria expressions are given as follows:

$$MAPE=\frac{1}{N_T}{\displaystyle \sum _{i=1}^{N_T}}\left|(y_i-{\widehat{y}}_i)/y_i\right|\times 100\%$$

(31)

$$MAE=\frac{1}{N_T}{\displaystyle \sum _{i=1}^{N_T}}\left|y_i-{\widehat{y}}_i\right|$$

(32)

$$MAXE=max\left|y_i-{\widehat{y}}_i\right|$$

(33)

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^{N_T}}{\left(y_i-{\widehat{y}}_i\right)}^2}{{\displaystyle \sum _{i=1}^{N_T}}{\left(y_i-{\overline{y}}_i\right)}^2},\overline{y}=\frac{1}{N_T}{\displaystyle \sum _{i=1}^{N_T}}{y}_i$$

(34)

The four performance criteria: RMSE, MAPE, MAE, and MAXE are applied to estimate the accuracy performance of the models. $R^2$ denotes the degree of matching between the estimated and actual values. The smaller the values of RMSE, MAPE, MAE and MAXE are and the closer $R^2$ is to 1, the better the accuracy performance of a model.

The performance comparison results are shown in

Table 3. Performance comparison of the fCaO content online prediction models.

Models	Training Time/s	Testing Time/s	RMSE	MAPE/%	MAE	MAXE	${\mathit{R}}^{\mathbf{2}}$
OS-ELM	0.5327	0.0063	0.3182	20.46	0.1537	1.6725	0.9685
OS-KELM	0.7286	0.0152	0.1359	17.58	0.1257	1.2547	0.9729
COS-MKELM	0.3856	0.0041	0.1018	9.54	0.0758	1.0174	0.9798

. Compared with actual data, the simulation results of the fCaO online prediction model with three models for testing data are shown in Figure 5, Figure 6 and Figure 7.

As seen from the time data in Table 3, the training time and testing time of fCaO model based on COS-MKELM are smallest. The RMES, MAPE, MAE, and MAXE performance criteria values of fCaO model based on COS-MKELM are the least, and the criterion $R^2$ value is the closest to 1 among the three comparison models. As seen from the precision index data in Table 3, the predicted values of the cement clinker fCao content online prediction model based on the COS-MKELM algorithm are closer to the actual value of cement clinker fCaO content obtained by the laboratory test. In addition, the established cement clinker fCao content online prediction model has strong nonlinear identification ability, which meets the requirements of prediction accuracy of the fCaO content in the cement clinker firing process.

The simulation results in Figure 5, Figure 6 and Figure 7 show that the predicted fCaO content values based on the COS-MKELM algorithm are closer to the actual values than the other comparison algorithms. In conclusion, although the COS-MKELM algorithm has fewer initial samples in the simulation modeling process, the learning performance of COS-MKELM algorithm is improved greatly through the continuous arrival of sample data. The online prediction model of cement clinker fCaO content based on the proposed COS-MKELM algorithm shows better performance in both test accuracy and generalization ability, indicating that the cement clinker fCaO content online prediction model based on the proposed COS-MKELM algorithm is effective.

In short, the simulation results of the cement clinker fCaO content online prediction model using actual data show that the proposed COS-MKELM algorithm, which is based on the fusion of three kernel functions method and online sequential learning method, shows fast learning efficiency. In addition, the prediction model of cement clinker fCaO content has better nonlinear identification ability, regression accuracy, and generalization ability.

Therefore, the fCaO content value can be predicted by the fCaO content online prediction model based on the proposed COS-MKELM algorithm. When the new training data added or the production conditions of cement clinker changed during the online prediction of fCaO content, the parameters of fCaO model can be online sequential updated to adapt the current condition. With the continuous input of actual data, the characteristics of cement clinker fCaO content online prediction model are improved, so that the online prediction model can adapt to different production conditions and operate stably for a long time. Therefore, this paper provides a new idea for realizing the prediction of cement clinker fCaO content. Furthermore, according to the fCaO content predicted value, the operators and engineers can adjust the cement production process variables. In this way, the qualified rate of cement clinker can be guaranteed, and the energy consumption and the exhaust emission of cement clinker production can be reduced.

5. Conclusions

In this paper, the COS-MKELM algorithm is proposed to avoid the large matrix inverse calculation and reduce the computational complexity of online sequential learning process. Three regression datasets were used to test the performance of the COS-MKELM algorithm. Experiment results show that the proposed COS-MKELM algorithm, which is based on the fusion of the three kernel functions method and the online sequential learning method, shows fast learning efficiency. Although the COS-MKELM algorithm has fewer initial samples in the simulation modeling process, the learning performance of COS-MKELM algorithm is improved greatly through the continuous arrival of sample data.

In addition, online prediction fCaO content is crucial and useful to the cement process industry. The proposed COS-MKELM algorithm is adopted to build the cement clinker fCaO content online prediction model of rotary kiln with 5000 tons of clinker production capacity per day in this paper. It can be seen from experiment results that the online prediction model of cement clinker fCaO content based on the proposed COS-MKELM algorithm shows better performance in nonlinear identification ability, regression accuracy, and generalization ability. The characteristics of cement clinker fCaO content online prediction model can be improved with the iteration update of cement calcination process data. Therefore, this research provides a new idea for realizing the prediction of cement clinker fCaO content.

Based on the online prediction model proposed in this research, the cement clinker fCaO content can be mastered by operators and engineers in real time. This is beneficial to the clinker quality and the energy consumption of the cement calcination process. Furthermore, the fCaO online sequential model built in this paper provides the necessary prerequisite for the cement process industry to build the intelligent control system and minimize energy costs.