# Fault Detection and Identification with Kernel Principal Component Analysis and Long Short-Term Memory Artificial Neural Network Combined Method

^{1}

^{2}

^{*}

## Abstract

**:**

^{2}and Q. The K-means clustering algorithm is then adopted to analyze the data and perform clustering, according to the type of fault. Finally, the type of fault is determined using a long short-term memory (LSTM) neural network. The performance of the proposed technique is compared with the principal component analysis (PCA) method in early detecting malfunctions on a continuous stirred tank reactor (CSTR) system. Up to 10 sensor faults and other system degradation conditions are considered. The performance of the LSTM neural network is compared with three other machine learning techniques, namely the support vector machine (SVM), K-nearest neighbors (KNN) algorithm, and decision trees, in determining the type of fault. The results indicate the superior performance of the suggested methodology in both early fault detection and fault identification.

## 1. Introduction

- (1)
- Process or component fault: Process faults arise when a system’s components behave negatively, affecting the dynamics of the system.
- (2)
- Actuator fault: An actuator fault is a discrepancy between the actuator’s input command and its actual output.
- (3)

## 2. Literature Review

- The KPCA is performed to decrease the dimension of the original data set while detecting the existence of potential faults. Thus, in subsequent fault identification steps, computational burden and transmission energy consumption is reduced, which is very important in wireless sensor networks.
- In the reduced data space, the K-means is used for clustering data into different groups and detecting faults using statistics. By using clustering, faults in different processes can be detected, which is an advantage typically overlooked in traditional approaches.
- An LSTM network is trained in order to identify faults by reconstruction. The LSTM excels at capturing temporal dependencies in sequential data, identifying fault patterns that unfold over time. This temporal modeling capability enhances fault identification and enables the detection of complex and dynamic fault scenarios.
- Simulations demonstrate the effectiveness of the method in the detection and identification of faults. Thus, based on measured data, the method can identify crashed sensors and actuators and components’ misbehavior.

## 3. The Proposed Method

- The KPCA for fault detection.
- The K-means clustering for processing faulty data.
- The LSTM neural network for determining the fault type.

#### 3.1. Fault Detection by KPCA

#### 3.2. Faulty Data Labeling Using the K-Means Technique for Clustering

- Utilizes an iterative technique to choose the greatest value for K center points.
- Chooses the closest K center for each data point. A cluster is formed by the data points that are near the K’s center.

#### 3.3. Using an LSTM Neural Network to Identify the Type of Fault

#### 3.4. SVM-Based Fault Type Identification

#### 3.5. KNN-Based Fault Type Identification

#### 3.6. Decision Trees Based Fault Type Identification

## 4. Description of the CSTR System

^{−1}, tank volume V = 150.0 L, jacket volume V

_{c}= 10.0 L, heat of reaction ∆H

_{r}= −0.2 × 10

^{5}cal·mol

^{−1}, heat transfer coefficient UA = 7.0 × 10

^{5}cal·min

^{−1}·K

^{−1}, pre-exponential factor to k k

_{0}= 7.2 × 10

^{10}min

^{−1}, activation energy E/R = 10

^{4}K, fluid density ρ, ρ

_{c}= 1000.0 g·L

^{−1}, and fluid heat capacity C

_{p}, C

_{pc}= 1.0 cal·g

^{−1}·K

^{−1}.

## 5. Simulation Results

## 6. Conclusions

- In the sample where the fault appears, the KPCA fault detection method finds the existing fault very well and with excellent accuracy.
- The KPCA approach minimizes the data dimensions, sometimes even to half of the real dimensions, which decreases the number of calculations and speeds up computer processing.
- Unlike the PCA approach, the KPCA method finds faults in all data and performs well for nonlinear data.
- When faults are classified using K-means clustering, the identification is substantially more accurate.
- Due to its recursion, the LSTM network produces relatively significant results when compared to other machine learning techniques.
- The LSTM network correctly identifies faults, and its accuracy is very high, reaching 99.09%.
- Naturally, the proposed approach can be applied to detect and identify faults in other types of equipment.

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Hwang, I.; Kim, S.; Kim, Y.; Seah, C.E. A survey of fault detection, isolation, and reconfiguration methods. IEEE Trans. Control Syst. Technol.
**2010**, 18, 636–653. [Google Scholar] [CrossRef] - Thirumarimurugan, M.; Bagyalakshmi, N.; Paarkavi, P. Comparison of Fault Detection and Isolation Methods: A Review. In Proceedings of the IEEE 10th International Conference on Intelligent Systems and Control (ISCO), Coimbatore, India, 7–8 January 2016. [Google Scholar]
- Khalili, M.; Zhang, X.; Cao, Y.; Polycarpou, M.M.; Parisini, T. Distributed adaptive fault-tolerant control of nonlinear uncertain second-order multi-agent systems. In Proceedings of the IEEE Annual Conference on Decision and Control (CDC), Osaka, Japan, 15–18 December 2015. [Google Scholar]
- Kersten, J.; Rauh, A.; Aschemann, H. Analyzing Uncertain Dynamical Systems after State-Space Transformations into Cooperative Form: Verification of Control and Fault Diagnosis. Axioms
**2021**, 10, 88. [Google Scholar] [CrossRef] - Muradore, R.; Fiorini, P. A PLS-Based Statistical Approach for Fault Detection and Isolation of Robotic Manipulators. IEEE Trans. Ind. Electron.
**2012**, 59, 3167–3175. [Google Scholar] [CrossRef] - Deng, X.; Tian, X.; Chen, S.J.; Harris, C. Nonlinear Process Fault Diagnosis Based on Serial Principal Component Analysis. IEEE Trans. Neural Netw. Learn. Syst.
**2018**, 29, 560–572. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Zidi, S.; Moulahi, T.; Alaya, B. Fault detection in Wireless Sensor Networks through SVM classifier. IEEE Sens. J.
**2017**, 6, 340–347. [Google Scholar] [CrossRef] - Kiranyaz, S.; Gastli, A.; Ben-brahim, L. Real-Time Fault Detection and Identification for MMC using 1-D Convolutional Neural Networks. IEEE Trans. Ind. Electron.
**2019**, 66, 8760–8771. [Google Scholar] [CrossRef] - Krishnamoorthy, G.; Ashok, P.; Tesar, D. Simultaneous Sensor and Process Fault Detection and Isolation in Multiple-Input—Multiple-Output Systems. IEEE Syst. J.
**2015**, 9, 335–349. [Google Scholar] [CrossRef] - Rohani, R.; Koochaki, A. A Hybrid Method Based on Optimized Neuro-Fuzzy System and Effective Features for Fault Location in VSC-HVDC Systems. IEEE Access
**2020**, 8, 70861–70869. [Google Scholar] [CrossRef] - Zhou, Z.; Wen, C.; Yang, C. Fault Isolation Based on K-Nearest Neighbor Rule for Industrial Processes. IEEE Trans. Ind. Electron.
**2016**, 63, 2578–2586. [Google Scholar] [CrossRef] - Fezai, R.; Nounou, M.; Messaoud, H. Reliable Fault Detection and Diagnosis of Large-Scale Nonlinear Uncertain Systems Using Interval Reduced Kernel. IEEE Access
**2020**, 8, 78343–78353. [Google Scholar] [CrossRef] - Cai, L.; Tian, X.; Chen, S. Monitoring Nonlinear and Non-Gaussian Processes Using Gaussian Mixture Model-Based Weighted Kernel Independent Component Analysis. IEEE Trans. Neural Netw. Learn. Syst.
**2015**, 28, 122–135. [Google Scholar] [CrossRef] [Green Version] - Jiang, Q.; Yan, X. Multimode Process Monitoring Using Variational Bayesian Inference and Canonical Correlation Analysis. IEEE Trans. Autom. Sci. Eng.
**2019**, 16, 1814–1824. [Google Scholar] [CrossRef] - Li, W.; Monti, A.S.; Ponci, F. Fault Detection and Classification in Medium Voltage DC Shipboard Power Systems With Wavelets and Artificial Neural Networks. IEEE Trans. Instrum. Meas.
**2014**, 63, 2651–2665. [Google Scholar] [CrossRef] - Ni, J.; Zhang, C.; Yang, S. An Adaptive Approach Based on KPCA and SVM for Real-Time Fault Diagnosis of HVCBs. IEEE Trans. Power Deliv.
**2011**, 26, 1960–1971. [Google Scholar] [CrossRef] - Pilario, K.E.S.; Cao, Y.; Shafiee, M. Mixed kernel canonical variate dissimilarity analysis for incipient fault monitoring in nonlinear dynamic processes. Comput. Chem. Eng.
**2019**, 123, 143–154. [Google Scholar] [CrossRef] [Green Version] - Lee, J.M.; Yoo, C.K.; Choi, S.W.; Vanrolleghem, P.A.; Lee, I.B. Nonlinear process monitoring using kernel principal component analysis. Chem. Eng. Sci.
**2004**, 59, 223–234. [Google Scholar] [CrossRef] - Likas, A.; Vlassis, N.; Verbeek, J.J. The global K-means clustering algorithm. Pattern Recognit.
**2003**, 36, 451–461. [Google Scholar] [CrossRef] [Green Version] - Shang, Y.; Zhou, B.; Wang, Y.; Li, A.; Chen, K.; Song, Y.; Lin, C. Popularity Prediction of Online Contents via Cascade Graph and Temporal Information. Axioms
**2021**, 10, 159. [Google Scholar] [CrossRef] - Qiao, M.; Yan, S.; Tang, X.; Xu, C. Deep Convolutional and LSTM Recurrent Neural Networks for Rolling Bearing Fault Diagnosis under Strong Noises and Variable Loads. IEEE Access
**2020**, 8, 66257–66269. [Google Scholar] [CrossRef] - Park, D.; Kim, S.; An, Y.; Jung, J. LiReD: A Light-Weight Real-Time Fault Detection Neural Networks. Sensors
**2018**, 18, 2110. [Google Scholar] [CrossRef] [Green Version] - Elsheikh, A.; Yacout, S.; Ouali, M. Bidirectional Handshaking LSTM for Remaining Useful Life Prediction. Neurocomputing
**2018**, 323, 148–156. [Google Scholar] [CrossRef] - Stoean, C.; Zivkovic, M.; Bozovic, A.; Bacanin, N. Metaheuristic-Based Hyperparameter Tuning for Recurrent Deep Learning: Application to the Prediction of Solar Energy Generation. Axioms
**2023**, 12, 266. [Google Scholar] [CrossRef] - Shi, Z.; Chehade, A. A Dual-LSTM Framework Combining Change Point Detection and Remaining Useful Life Prediction. Reliab. Eng. Syst. Saf.
**2020**, 205, 107257. [Google Scholar] [CrossRef] - Jeong, K.; Choi, S.B.; Choi, H. Sensor Fault Detection and Isolation Using a Support Vector Machine for Vehicle Suspension Systems. IEEE Trans. Veh. Technol.
**2020**, 69, 3852–3863. [Google Scholar] [CrossRef] - Lee, C.; Alena, R.L.; Robinson, P. Migrating fault trees to decision trees for real time fault detection on international space station. In Proceedings of the IEEE Aerospace Conference, Big Sky, MT, USA, 5–12 March 2005. [Google Scholar]
- Yang, X.; Wei, Q. Adaptive Critic Designs for Optimal Event-Driven Control of a CSTR System. IEEE Trans. Ind. Inform.
**2021**, 17, 484–493. [Google Scholar] [CrossRef] - Pilario, K.E.S.; Cao, Y. Canonical variate dissimilarity analysis for process incipient fault detection. IEEE Trans. Ind. Inform.
**2018**, 14, 5308–5315. [Google Scholar] [CrossRef] [Green Version] - Zhuang, L.; Xu, A.; Wang, W.L. A prognostic driven predictive maintenance framework based on Bayesian deep learning. Reliab. Eng. Syst. Saf.
**2023**, 234, 109181. [Google Scholar] [CrossRef]

**Figure 2.**Overview of the CSTR system [29].

**Table 1.**Scenarios of CSTR primary faults [29].

Number | Designation | Description | Progression Rate γ | Type |
---|---|---|---|---|

1 | Catalyst decay | ${a=a}_{0}\mathrm{exp}(-\mathsf{\gamma}t)$ | 0.0005 | Multiplicative |

2 | Fouling | $b={b}_{0}\mathrm{exp}(-\mathsf{\gamma}t)$ | 0.001 | Multiplicative |

3 | Simultaneous | faults 1 and 2 | - | Multiplicative |

4 | Sensor | ${C}_{i}{=C}_{i,0}+\mathsf{\gamma}t$ | 0.001 | Additive |

5 | Sensor | ${T}_{i}{=T}_{i,0}+\mathsf{\gamma}t$ | 0.05 | Additive |

6 | Sensor | ${T}_{\mathrm{Ci}}{=T}_{\mathrm{Ci},0}+\mathsf{\gamma}t$ | 0.05 | Additive |

7 | Sensor | ${C=C}_{0}+\mathsf{\gamma}t$ | 0.001 | Additive |

8 | Sensor | ${T=T}_{0}+\mathsf{\gamma}t$ | 0.05 | Additive |

9 | Sensor | ${T}_{C}{=T}_{C,0}+\mathsf{\gamma}t$ | 0.05 | Additive |

10 | Sensor | ${Q}_{C}{=Q}_{C,0}+\mathsf{\gamma}t$ | −0.1 | Additive |

Fault | Faulty Samples | $\mathbf{Fault}\mathbf{Detection}\mathbf{Sample}\mathbf{by}{\mathit{T}}^{2}$ | Fault Detection Sample by Q |
---|---|---|---|

1 | 151–210 | 151 | 151 |

2 | 121–300 | 122 | 121 |

3 | 71–140 | 73 | 71 |

4 | 131–300 | 132 | 131 |

5 | 71–250 | 75 | 72 |

6 | 61–280 | 62 | 61 |

7 | 121–270 | 123 | 121 |

8 | 111–300 | 111 | 111 |

9 | 151–300 | 152 | 151 |

10 | 151–300 | 153 | 151 |

Various Implementations of the LSTM Network | Fault Type | Precision % | Recall % | Accuracy % |
---|---|---|---|---|

No clustering | 1 | 80.6 | 79.1 | 81.5 |

2 | 77.3 | 83.5 | ||

3 | 81.4 | 76.9 | ||

4 | 75.3 | 80.1 | ||

5 | 83.2 | 77.6 | ||

6 | 81.1 | 82.6 | ||

7 | 79.9 | 80.8 | ||

8 | 83.2 | 79.8 | ||

9 | 78.5 | 79.3 | ||

10 | 82.03 | 80.6 | ||

Clustering all data at once | 1 | 86.6 | 88.1 | 89 |

2 | 90.5 | 91.3 | ||

3 | 89.3 | 87.9 | ||

4 | 90.2 | 89.1 | ||

5 | 87.4 | 88.9 | ||

6 | 84.8 | 87.3 | ||

7 | 91.6 | 93.2 | ||

8 | 90.6 | 90.3 | ||

9 | 88.7 | 89.5 | ||

10 | 89.9 | 90.5 | ||

Data divided into three categories, with individual clustering for each category | 1 | 100 | 100 | 99.09 |

2 | 98.6 | 99.5 | ||

3 | 99.01 | 99.3 | ||

4 | 100 | 100 | ||

5 | 99.3 | 99.06 | ||

6 | 98.9 | 99.07 | ||

7 | 99.5 | 98.8 | ||

8 | 99.1 | 98.06 | ||

9 | 99.00 | 99.4 | ||

10 | 99.8 | 99.2 |

**Table 4.**Comparing the accuracy, recall, and precision of different machine learning methods for fault identification.

Various Machine Learning Methods | Fault Type | Precision % | Recall % | Accuracy % |
---|---|---|---|---|

SVM | 1 | 21.2 | 22.0 | 23 |

2 | 22.3 | 25.8 | ||

3 | 26.5 | 28.7 | ||

4 | 20.4 | 21.3 | ||

5 | 24.1 | 20.1 | ||

6 | 25.3 | 26.7 | ||

7 | 22.2 | 23.2 | ||

8 | 24.2 | 24.6 | ||

9 | 21.1 | 23.2 | ||

10 | 22.8 | 21.3 | ||

KNN | 1 | 27.1 | 28.2 | 27.5 |

2 | 28.3 | 27.3 | ||

3 | 26.5 | 26.2 | ||

4 | 29.9 | 25.1 | ||

5 | 28.7 | 24.6 | ||

6 | 27.3 | 29.1 | ||

7 | 27.8 | 23.7 | ||

8 | 27.1 | 28.2 | ||

9 | 29.0 | 27.1 | ||

10 | 26.5 | 24.5 | ||

Decision Tree | 1 | 41.3 | 46.2 | 45.2 |

2 | 42.2 | 48.6 | ||

3 | 39.8 | 41.9 | ||

4 | 48.9 | 44.3 | ||

5 | 45.5 | 46.1 | ||

6 | 44.3 | 40.2 | ||

7 | 49.3 | 41.8 | ||

8 | 51.2 | 48.1 | ||

9 | 55.1 | 49.2 | ||

10 | 43.9 | 40.8 | ||

LSTM | 1 | 100 | 100 | 99.09 |

2 | 98.6 | 99.5 | ||

3 | 99.01 | 99.3 | ||

4 | 100 | 100 | ||

5 | 99.3 | 99.06 | ||

6 | 98.9 | 99.07 | ||

7 | 99.5 | 98.8 | ||

8 | 99.1 | 98.06 | ||

9 | 99.00 | 99.4 | ||

10 | 99.8 | 99.2 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jafari, N.; Lopes, A.M.
Fault Detection and Identification with Kernel Principal Component Analysis and Long Short-Term Memory Artificial Neural Network Combined Method. *Axioms* **2023**, *12*, 583.
https://doi.org/10.3390/axioms12060583

**AMA Style**

Jafari N, Lopes AM.
Fault Detection and Identification with Kernel Principal Component Analysis and Long Short-Term Memory Artificial Neural Network Combined Method. *Axioms*. 2023; 12(6):583.
https://doi.org/10.3390/axioms12060583

**Chicago/Turabian Style**

Jafari, Nahid, and António M. Lopes.
2023. "Fault Detection and Identification with Kernel Principal Component Analysis and Long Short-Term Memory Artificial Neural Network Combined Method" *Axioms* 12, no. 6: 583.
https://doi.org/10.3390/axioms12060583