# Intelligent Fault Prognosis Method Based on Stacked Autoencoder and Continuous Deep Belief Network

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- (1)
- In order to construct a health index that can accurately represent the change in model performance, this paper makes full use of the powerful feature learning capabilities and information fusion capabilities of stacked autoencoders.
- (2)
- In order to solve the problem of local random interference in the health index, this paper uses the exponential weighted moving average method to smooth the health index. It can integrate historical information with current information to reduce the impact of noise.
- (3)
- In order to capture the feature information of the continuous time series, this paper makes full use of the continuous deep belief network’s powerful feature learning ability and fault prognosis ability for continuous data.
- (4)
- In order to verify the prediction effect of the fault prognosis method in this paper, a comparative experiment on fault prognosis is carried out using intelligent maintenance systems (IMS) bearing data as an example.

## 2. Basic Theories of Stacked Autoencoder and Deep Belief Network

#### 2.1. Stacked Autoencoder

#### 2.2. Deep Belief Network

## 3. Proposed Method

#### 3.1. Construction of Health Index

#### 3.2. The Continuous Deep Belief Network

#### 3.3. The Fault Prognosis Steps Based on the Proposed Method

- (1)
- Data collection. A bearing performance test experiment is designed and sensors are used to collect bearing lifecycle vibration information.
- (2)
- Establish health indices. Firstly, feature parameters containing information on bearing performance changes are extracted from the vibration signals collected by the sensors. Then, the stacked autoencoder is used to perform feature learning and information fusion on the extracted feature parameters to construct the health indices that characterize the performance change in the model. Aiming at the local fluctuations in health indices caused by noise, the data are smoothed using the exponentially weighted moving average method to reduce the impact of noise.
- (3)
- Feature learning for continuous deep belief networks. The model performance change index is a continuous time series. Firstly, a CDBN model suitable for processing continuous data is constructed. Then, its own powerful feature learning ability and information mining ability is used to capture the performance change trends hidden in health indices, and model fault prognosis is performed.
- (4)
- Visualization of failure results. The fault prognosis results are visualized.

## 4. Data Processing and Experiment

#### 4.1. The Bearing Data Introduction

#### 4.2. The Health Indices Construction

- (1)
- Compared with a single characteristic parameter model to characterize the performance of the model, the health indices constructed by information fusion of multiple characteristic parameters can more accurately reflect the performance changes of the model.
- (2)
- Compared with machine learning methods such as PCA and kernel principal component analysis (KPCA), stacked autoencoders as deep learning models can better perform feature learning and information fusion on the extracted feature parameters, and the health indices constructed are more accurate.
- (3)
- The exponentially weighted moving average can reduce the impact of noise on health indices, and can capture mutation information while maintaining the overall trend of the model. The health indices constructed by combining the advantages of the SAE and the EWMA can better characterize the performance changes in the model and the tendency of the bearing performance to degrade compared to a single characteristic value.

#### 4.3. The Health Indices Fault Prognosis

#### 4.3.1. Data Processing

#### 4.3.2. The CDBN Parameter Determination Experiment

- (1)
- The feature learning ability and information fusion ability of a single CRBM model are limited. The CDBN model uses the greedy algorithm to combine multiple CRBM models. The feature learning ability and non-linear approximation ability are stronger and the method is more suitable for dealing with complex non-linear prediction problems.
- (2)
- A model with a more complicated network structure will not have a stronger failure prediction ability. This is because the more complex the network structure, the stronger the ability to learn the features of the model, which may lead to over-learning of the training samples. At the same time as learning the performance of the model, some interference information may be learned, which affects the results of fault prediction.

#### 4.3.3. Fault Prognosis Comparative Experiments

- (1)
- BPNN: The network structure was 4-24-1. The learning rate, momentum, and iteration time were 0.01, 0.9, and 500, respectively.
- (2)
- SVR: The lag order was four and the penalty parameter and kernel radius were 30 and 0.01, respectively, chosen by 5-fold cross validation. The Gaussian kernel was used.
- (3)
- DBN: The network structure was 4-12-8-1. The learning rate, momentum, and iteration time were 0.11, 0.9, and 200, respectively.
- (4)
- LSTM: The network structure was 4-12-8-1. The learning rate, momentum, and iteration time were 0.01, 0.9, and 500, respectively. The network was constructed with a logistic regression layer.
- (5)
- RNN: The network structure was 4-12-8-1. The learning rate, momentum, and iteration time were 0.37, 0.9, and 700, respectively.

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

SAE | Stacked autoencoder |

EWMA | Exponentially weighted moving average |

PCA | Principal component analysis |

LLE | Locally linear embedding |

IMS | Intelligent maintenance systems |

DBN | Deep belief network |

RBM | Restricted Boltzmann machine |

BP | Back propagation |

DBN | Deep belief network |

MCD | Minimizing contrastive divergence |

CRBM | Continuity restricted Boltzmann machine |

CDBN | Continuous deep belief network |

KPCA | Kernel principal component analysis |

RMSE | Root mean square error |

MaxAE | Maximum absolute value error |

MAE | Mean absolute value error |

SVR | Support vector regression |

LSTM | Long short-term memory |

RNN | Recurrent neural network |

## References

- Tian, F.; Luo, R.; Jia, L. Non-Stationary Feature Extraction Method for Mechanical Faults and Its Application; National Defense Industry Press: Beijing, China, 2014. [Google Scholar]
- Peng, Y.; Dong, M. A prognosis method using age-dependent hidden semi-Markov model for equipment health prediction. Mech. Syst. Signal Process.
**2011**, 25, 237–252. [Google Scholar] [CrossRef] - Yu, M.; Xiao, C.; Wang, H.; Jiang, W.; Zhu, R. Adaptive Cuckoo Search-Extreme Learning Machine Based Prognosis for Electric Scooter System under Intermittent Fault. Actuators
**2021**, 10, 283. [Google Scholar] [CrossRef] - Yu, M.; Lu, H.; Wang, H.; Xiao, C.; Lan, D.; Chen, J. Computational intelligence-based prognosis for hybrid mechatronic system using improved wiener process. Actuators
**2021**, 10, 213. [Google Scholar] [CrossRef] - Janjarasjitt, S.; Ocak, H.; Loparo, K. Bearing condition diagnosis and prognosis using applied nonlinear dynamical analysis of machine vibration signal. J. Sound Vib.
**2008**, 317, 112–126. [Google Scholar] [CrossRef] - Lei, Y.; Liu, Z.; Lin, J.; Lu, F. Phenomenological models of vibration signals for condition monitoring and fault diagnosis of epicyclic gearboxes. J. Sound Vib.
**2016**, 369, 266–281. [Google Scholar] [CrossRef] - Huang, D.; Huang, X.; Fan, M.; Xiong, Z. Mixed fault prediction based on Kalman filtering and expert system. Comput. Simul.
**2005**, 22, 150–152. [Google Scholar] - Du, Z.; Li, X.; Zheng, Z.; Mao, Q. Fault prediction with combination of strong tracking square-root cubature Kalman filter and autoregressive model. Control Theory Appl.
**2014**, 31, 1047–1052. [Google Scholar] - Lin, P.; Wang, K. Particle Filter Fault Prediction Based on Fuzzy Closeness Degree. Appl. Comput. Syst.
**2017**, 26, 134–138. [Google Scholar] - Baptista, M.; de Medeiros, I.P.; Malere, J.P.; Nascimento, C.; Prendinger, H.; Henriques, E.M. Comparative case study of life usage and data-driven prognostics techniques using aircraft fault messages. Comput. Ind.
**2017**, 86, 1–14. [Google Scholar] [CrossRef] - Civera, M.; Surace, C. Non-destructive techniques for the condition and structural health monitoring of wind turbines: A literature review of the last 20 years. Sensors
**2022**, 22, 1627. [Google Scholar] [CrossRef] - Zhang, P.; Zhou, L.; Zhang, Z.; Li, J. Analysis and Prediction of Transformer Overheating Fault Based on Chaotic Characteristics of Oil Chromatogram. High Volt. Appar.
**2019**, 55, 237–243. [Google Scholar] - Peng, G.; Zhou, Z.; Tang, S.; Wu, T.; Wu, X. Transformer Fault Prediction Based on Timing Analysis and Variable Correction. Electron. Meas. Technol.
**2018**, 41, 96–99. [Google Scholar] - Yan, J.; Koç, M.; Lee, J. A prognostic algorithm for machine performance assessment and its application. Prod. Plan. Control
**2004**, 15, 796–801. [Google Scholar] [CrossRef] - Rigamonti, M.G.; Baraldi, P.; Zio, E.; Roychoudhury, I.; Goebel, K.; Poll, S. Ensemble of optimized echo state networks for remaining useful life prediction. Neurocomputing
**2017**, 281, 121–138. [Google Scholar] [CrossRef] [Green Version] - Xu, J.; Liu, L.; Zhang, L.; Duan, W.; Liu, S. Research on multi-label prediction model of shield fault based on PCA-LSTM. J. Shandong Agric. Univ. (Nat. Sci. Ed.)
**2019**, 50, 1005–1009. [Google Scholar] - Li, X.; Jiang, H.; Xiong, X.; Shao, H. Rolling bearing health prognosis using a modified health index based hierarchical gated recurrent unit network. Mech. Mach. Theory
**2019**, 133, 229–249. [Google Scholar] [CrossRef] - Civera, M.; Surace, C. An application of instantaneous spectral entropy for the condition monitoring of wind turbines. Appl. Sci.
**2022**, 12, 1059. [Google Scholar] [CrossRef] - Zhou, F.; Gao, Y.; Wang, J.; Wen, C. Early diagnosis and life prognosis for slowly varying fault based on deep learning. J. Shandong Univ. (Eng. Sci. Ed.)
**2017**, 47, 30–37. [Google Scholar] - Hinton, G.E.; Osindero, S.; Teh, Y.W. A Fast Learning Algorithm for Deep Belief Nets. Neural Comput.
**2006**, 18, 1527–1554. [Google Scholar] [CrossRef] - Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw.
**2015**, 61, 85–117. [Google Scholar] [CrossRef] [Green Version] - Guo, L.; Li, N.; Jia, F.; Lei, Y.; Lin, J. A recurrent neural network based health indicator for remaining useful life prediction of bearings. Neurocomputing
**2017**, 240, 98–109. [Google Scholar] [CrossRef] - Chen, H.; Murray, A.F. Continuous restricted Boltzmann machine with an implementable training algorithm. IEE Proc.-Vis. Image Signal Process.
**2003**, 150, 153–158. [Google Scholar] [CrossRef] [Green Version] - Takens, F. Detecting Strange Attractors in Turbulence. In Dynamical Systems and Turbulence, Warwick 1980; Springer: Berlin/Heidelberg, Germany, 1981. [Google Scholar]

**Figure 7.**The health indices constructed by the SAE, where the right subgraph is a zoomed graph of the left subfigure in the range of 15,000∼18,000 min.

**Figure 8.**The health indices constructed by the EWMA and the SAE, where the right subgraph is a zoomed graph of the left subfigure in the range of 15,000∼18,000 min.

**Figure 9.**The envelope spectrum of the vibration signal collected by the sensor during different periods. (

**a**) Envelope spectrum for a bearing running at 14,690 min. (

**b**) Envelope spectrum for a bearing running at 15,710 min. (

**c**) Envelope spectrum for a bearing running at 15,720 min. (

**d**) Envelope spectrum for a bearing running at 18,200 min.

**Figure 11.**The results of different fault prognosis methods. (

**a**) The fault prognosis results of the CDBN. (

**b**) The fault prognosis results of the BPNN. (

**c**) The fault prognosis results of the SVR. (

**d**) The fault prognosis results of the DBN. (

**e**) The fault prognosis results of the LSTM. (

**f**) The fault prognosis results of the RNN.

Signal Characteristic | Calculation Formula |
---|---|

Root mean square | ${X}_{\mathrm{ms}}=\sqrt{\frac{1}{N}{\sum}_{\mathrm{i}=1}^{N}{X}_{i}^{2}}$ |

Average value | ${X}_{\mathrm{ms}}=\frac{1}{N}{\sum}_{\mathrm{i}=1}^{N}{X}_{i}$ |

Variance | ${X}_{\mathrm{var}}=\frac{1}{N}{\sum}_{i=1}^{N}{\left({X}_{i}-{X}_{\mathrm{mean}\phantom{\rule{4.pt}{0ex}}}\right)}^{2}$ |

Kurtosis | $K=\frac{1}{N}{\sum}_{i=1}^{N}{\left(\frac{{X}_{i}-\mu}{\sigma}\right)}^{4}$ |

Peak-to-peak value | ${X}_{pp}={X}_{max}-{X}_{min}$ |

Parameters | |
---|---|

Network structure | 8-15-6-2-1 |

Activation function | Sigmoid function |

The number of iterations | 30 |

The noise constant | 0.1 |

Vibration Signal | Eigenvalues | Spearman Coefficient |
---|---|---|

Health index | SAE+EWMA | 0.8738 |

SAE | 0.7527 | |

PCA | 0.7404 | |

KPCA | 0.7402 | |

Time domain | Averagee | 0.7101 |

Variance | 0.7259 | |

RMS | 0.7246 | |

Peak-to-peak | 0.5911 | |

Kurtosis | 0.6381 | |

Frequency domain | Average | 0.7042 |

Variance | 0.5784 | |

RMS | 0.7238 |

Operating Hours | Health Indices Stage |
---|---|

0∼15,710 min | The bearing fault has a negligible effect on the system |

15,720∼18,250 min | The bearing fault has a considerable effect on the system |

Number | CDBN Network Structure | RMSE (${10}^{-3}$) | MaxAE (${10}^{-2}$) | MAE (${10}^{-3}$) |
---|---|---|---|---|

1 | 4-8-1 | 6.3 ± 0.34 | 5.25 ± 0.142 | 3.9 ± 0.23 |

2 | 4-12-1 | 6.4 ± 0.42 | 5.31 ± 0.136 | 4.2 ± 0.26 |

3 | 4-16-1 | 6.2 ± 0.25 | 5.34 ± 0.153 | 4.1 ± 0.24 |

4 | 4-16-12-1 | 5.7 ± 0.43 | 5.21 ± 0.123 | 3.5 ± 0.31 |

5 | 4-16-8-1 | 5.9 ± 0.32 | 5.42 ± 0.142 | 3.6 ± 0.24 |

6 | 4-12-8-1 | 5.6 ± 0.41 | 4.98 ± 0.135 | 3.2 ± 0.19 |

7 | 4-12-4-1 | 5.7 ± 0.52 | 5.15 ± 0.138 | 3.4 ± 0.32 |

8 | 4-12-8-4-1 | 5.9 ± 0.51 | 5.24 ± 0.141 | 3.6 ± 0.36 |

9 | 4-16-8-4-1 | 6.0 ± 0.32 | 5.26 ± 0.156 | 3.7 ± 0.25 |

10 | 4-16-12-4-1 | 6.2 ± 0.43 | 5.35 ± 0.148 | 3.8 ± 0.43 |

Parameters | |
---|---|

Network structure | 4-12-8-1 |

Learning rate | 0.11 |

Iteration times | 100 |

Noise constant | 0.0001 |

Lower and upper asymptotes | −1, 1 |

Network | RMSE (${10}^{-3}$) | MaxAE (${10}^{-2}$) | MAE (${10}^{-3}$) |
---|---|---|---|

CDBN | 5.6 ± 0.41 | 4.98 ± 0.14 | 3.2 ± 0.19 |

BPNN | 12.1 ± 0.15 | 7 ± 0.52 | 5.9 ± 0.55 |

SVR | 7.7 | 10.78 | 6.8 |

DBN | 10.6 ± 0.72 | 9.1 ± 0.74 | 9.3 ± 0.5 |

LSTM | 7.3 ± 0.31 | 5.57 ± 0.25 | 5.6 ± 0.35 |

RNN | 6.8 ± 0.42 | 5.36 ± 0.22 | 4.4 ± 0.34 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhang, C.; Zhang, Y.; Huang, Q.; Zhou, Y.
Intelligent Fault Prognosis Method Based on Stacked Autoencoder and Continuous Deep Belief Network. *Actuators* **2023**, *12*, 117.
https://doi.org/10.3390/act12030117

**AMA Style**

Zhang C, Zhang Y, Huang Q, Zhou Y.
Intelligent Fault Prognosis Method Based on Stacked Autoencoder and Continuous Deep Belief Network. *Actuators*. 2023; 12(3):117.
https://doi.org/10.3390/act12030117

**Chicago/Turabian Style**

Zhang, Chao, Yibin Zhang, Qixuan Huang, and Yong Zhou.
2023. "Intelligent Fault Prognosis Method Based on Stacked Autoencoder and Continuous Deep Belief Network" *Actuators* 12, no. 3: 117.
https://doi.org/10.3390/act12030117