# A Novel RUL Prognosis Model Based on Counterpropagating Learning Approach

## Abstract

**:**

## 1. Introduction

## 2. Related Works

## 3. The Proposed Model

#### 3.1. Problem Formulation and Underlying Assumptions

#### 3.2. RGHSOM Unsupervised Layer

**,**are denoted by ${\mathit{w}}_{{m}_{j}}^{x}\left[k\right]$ and ${\mathit{w}}_{{m}_{j}}^{y}\left[k\right]$, respectively, where the ${\mathit{w}}_{{m}_{j}}^{x}\left[k\right]\in {\mathbb{R}}^{\mathrm{N}}$ and ${\mathit{w}}_{{m}_{j}}^{y}\left[k\right]\in {\mathbb{R}}^{\Vert \mathbf{M}\Vert}$. At this point, the map is presented with a realisation of the input, denoted $\mathcal{r}\left(t\right)\subseteq \mathcal{A}$, and each neuron calculates its distance with respect to $\mathcal{r}\left(t\right)$ as:

#### 3.3. Grossberg Layer Supervised Layer

## 4. Results and Discussion

#### 4.1. Overview of the C-MPASS Dataset

#### 4.2. Performance Metrics

#### 4.3. Assessment of the Model Learnability

#### 4.4. Assessment of the Model Evolving

#### 4.5. Comparison Prediction Accuracy with Related Works

#### 4.6. Comparison of Computational Complexity with Related Works

## 5. Conclusions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Research Nester. Global Aerospace Forging Market Size, Forecast and Trends Highlights Over 2023–2035. 2023. Available online: https://www.researchnester.com/reports/aerospace-forging-market/4104 (accessed on 3 October 2023).
- IATA. Airline Maintenance Cost Executive Commentary. 2022. Available online: https://www.iata.org/contentassets/bf8ca67c8bcd4358b3d004b0d6d0916f/fy2021-mctg-report_public.pdf (accessed on 3 October 2023).
- Sprong, J.; Jiang, X.; Polinder, H. Deployment of Prognostics to Optimize Aircraft Maintenance – A Literature Review. J. Int. Bus. Res. Mark.
**2020**, 5(4), 26–37. [Google Scholar] [CrossRef] - Uckun, S.; Goebel, K.; Lucas, P.J.F. Standardizing research methods for prognostics. In Proceedings of the 2008 International Conference on Prognostics and Health Management, Denver, CO, USA; 2008; pp. 1–10. [Google Scholar] [CrossRef]
- IATA. The Changing MRO Landscape. 2018. Available online: https://www.iata.org/contentassets/81005748740046de878439e6c54f2355/d1-1100-1130-the-changing-mro-landscape_icf.pdf (accessed on 3 October 2023).
- Rankin, W. MEDA Investigation Process; Technical Report; Boeing: Arlington, VA, USA, 2007. [Google Scholar]
- FAA Safety. Human Factors. Available online: https://www.faasafety.gov/files/gslac/courses/content/258/1097/AMT_Handbook_Addendum_Human_Factors.pdf (accessed on 3 October 2023).
- Kraft, J.; Sethi, V.; Singh, R. Optimization of Aero Gas Turbine Maintenance Using Advanced Simulation and Diagnostic Methods. J. Eng. Gas Turbines Power
**2014**, 136, 111601. [Google Scholar] [CrossRef] - Gerdes, M.; Galar, D.; Scholz, D. Effects of condition-based maintenance on costs caused by unscheduled maintenance of aircraft. J. Qual. Maint. Eng.
**2016**, 22, 394–417. [Google Scholar] [CrossRef] - Fritzsche, R.; Lasch, R. An Integrated Logistics Model of Spare Parts Maintenance Planning within the Aviation Industry. Int. J. Econ. Manag. Eng.
**2012**, 6, 1958–1967. [Google Scholar] - Rodrigues, L.R.; Yoneyama, T. Maintenance Planning Optimization Based on PHM Information and Spare Parts Availability. In Proceedings of the Annual Conference of the Prognostics and Health Management Society 2013, New Orleans, LA, USA, 14–17 October 2013; pp. 11–17. [Google Scholar]
- Daneshjo, N.; Majernik, M.; Danishjoo, E.; Krivosudska, J. Quality Management Maintenance of Air Traffic. Mod. Mach. Sci. J.
**2017**, 1845–1850. [Google Scholar] [CrossRef] - Yang, L.; Ma, X.B.; Zhao, Y. A condition-based maintenance model for a three-state system subject to degradation and environmental shocks. Comput. Ind. Eng.
**2017**, 105, 210–222. [Google Scholar] [CrossRef] - Chen, Y.X.; Gong, W.G.; Xu, D.; Kang, R. Imperfect maintenance policy considering positive and negative effects for deteriorating systems with variation of operating conditions. IEEE Trans. Autom. Sci. Eng.
**2017**, 15, 872–878. [Google Scholar] [CrossRef] - Austen, I.M.; McIntyre, P. Corrosion fatigue of high-strength steel in low-pressure hydrogen gas. Metal Sci.
**1979**, 13, 420–428. [Google Scholar] [CrossRef] - Ng, T.S. An application of the EM algorithm FP to degradation modeling. IEEE Trans. Rel.
**2008**, 1, 2–13. [Google Scholar] - Sun, W.; Wu, H.; Shi, L.; Yi, Y.; Li, H. On-resistance degradations for different stress conditions in high-voltage pLEDMOS transistor with thick gate oxide. IEEE Electron Device Let.
**2007**, 28, 631–633. [Google Scholar] - Doksum, K.A.; Hoyland, A. Models for variable-stress accelerated life testing experiments based on wiener processes and the inverse gaussian distribution. Theory Probab. Appliactions
**1993**, 37, 137–139. [Google Scholar] [CrossRef] - Kang, R.; Gong, W.; Chen, Y. Model-driven degradation modeling approaches: Investigation and review. Chin. J. Aeronaut.
**2020**, 33, 1137–1153. [Google Scholar] [CrossRef] - Ramasso, E.; Saxena, A. Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets. Int. J. Progn. Health Manag.
**2014**, 5, 1–15. [Google Scholar] [CrossRef] - Aggarwal, C.C. Neural Networks and Deep Learning: A Textbook, 1st ed.; Springer Nature: Cham, Switzerland, 2018. [Google Scholar]
- Asif, O.; Haider, S.A.; Naqvi, S.R.; Zaki, J.F.W.; Kwak, K.-S.; Islam, S.M.R. A Deep Learning Model for Remaining Useful Life Prediction of Aircraft Turbofan Engine on C-MAPSS Dataset. IEEE Access
**2022**, 10, 95425–95440. [Google Scholar] [CrossRef] - Liu, Y.; Liu, Z.; Zuo, H.; Jiang, H.; Li, P.; Li, X. A DLSTM-Network-Based Approach for Mechanical Remaining Useful Life Prediction. Sensors
**2022**, 22, 5680. [Google Scholar] [CrossRef] - Khumprom, P.; Grewell, D.; Yodo, N. Deep Neural Network Feature Selection Approaches for Data-Driven Prognostic Model of Aircraft Engines. Aerospace
**2020**, 7, 132. [Google Scholar] [CrossRef] - Sateesh Babu, G.; Zhao, P.; Li, X.L. Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life. In Database Systems for Advanced Applications; Navathe, S., Wu, W., Shekhar, S., Du, X., Wang, X., Xiong, H., Eds.; DASFAA 2016, Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2016; Volume 9642. [Google Scholar] [CrossRef]
- Xiang, L.; Ding, Q.; Sun, J.-Q. Remaining useful life estimation in prognostics using deep convolution neural networks. Reliab. Eng. Syst. Saf.
**2018**, 172, 1–11. [Google Scholar] - Hao, L.; Wang, Z.; Li, Z. An enhanced CNN-LSTM remaining useful life prediction model for aircraft engine with attention mechanism. PeerJ Comput. Sci.
**2022**, 8, e1084. [Google Scholar] - Maschler, B.; Vietz, H.; Jazdi, N.; Weyrich, M. Continual Learning of Fault Prediction for Turbofan Engines using Deep Learning with Elastic Weight Consolidation. In Proceedings of the 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Vienna, Austria, 8–11 September 2020; pp. 959–966. [Google Scholar] [CrossRef]
- Ali, B.; Björsell, N. Aggregation strategy on federated machine learning algorithm for collaborative predictive maintenance. Sensors
**2022**, 22, 6252. [Google Scholar] - Hu, Q.; Zhao, Y.; Wang, Y.; Peng, P.; Ren, L. Remaining Useful Life Estimation in Prognostics Using Deep Reinforcement Learning. IEEE Access
**2023**, 11, 32919–32934. [Google Scholar] [CrossRef] - Shi, J.; Gao, J.; Xiang, S. Adaptively Lightweight Spatiotemporal Information-Extraction-Operator-Based DL Method for Aero-Engine RUL Prediction. Sensors
**2023**, 23, 6163. [Google Scholar] [CrossRef] - Fitra, A.A.; Wijaya, S.K.; Naved, M. Prediction of remaining useful life using the CNN-GRU network: A study on maintenance management. Softw. Impacts
**2023**, 17, 100535. [Google Scholar] - Hu, Q.; Zhao, Y.; Ren, L. Novel Transformer-based Fusion Models for Aero-engine Remaining Useful Life Estimation. IEEE Access
**2023**, 11, 52668–52685. [Google Scholar] - Zhu, Q.; Xiong, Q.; Yang, Z.; Yu, Y. RGCNU: Recurrent Graph Convolutional Network with Uncertainty Estimation for Remaining Useful Life Prediction. IEEE/CAA J. Autom. Sin.
**2023**, 10, 1640–1642. [Google Scholar] [CrossRef] - Wang, Y.; Xu, Y.; Yang, J.; Wu, M.; Li, X.; Xie, L.; Chen, Z. Fully-Connected Spatial-Temporal Graph for Multivariate Time Series Data. arXiv
**2023**, arXiv:2309.05305. [Google Scholar] - Liu, Y.; Zhang, X.; Guo, W.; Bian, H.; He, Y.; Liu, Z. Prediction of remaining useful life of turbofan engine based on optimized model. In Proceedings of the 2021 IEEE 20th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Shenyang, China, 20–22 October 2021; pp. 1473–1477. [Google Scholar]
- Yu, K.; Wang, D.; Li, H. A Prediction Model for Remaining Useful Life of Turbofan Engines by Fusing Broad Learning System and Temporal Convolutional Network. In Proceedings of the 2021 8th International Conference on Information, Cybernetics, and Computational Social Systems (ICCSS), Beijing, China, 10–12 December 2021; pp. 137–142. [Google Scholar] [CrossRef]
- Indrė, Ž.; Pechenizkiy, M.; Gama, J. An overview of concept drift applications. In Big Data Analysis: New Algorithms for a New Society; Springer: Cham, Switzerland, 2016; pp. 91–114. [Google Scholar]
- Hoens, T.R.; Polikar, R.; Chawla, N.V. Learning from streaming data with concept drift and imbalance: An overview. Prog. Artif. Intell.
**2012**, 1, 89–101. [Google Scholar] [CrossRef] - Pascanu, R.; Mikolov, T.; Bengio, Y. On the difficulty of training recurrent neural networks. In Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA, 17–19 June 2013. [Google Scholar]
- Alzubaidi, L.; Zhang, J.; Humaidi, A.J.; Al-Dujaili, A.; Duan, Y.; Al-Shamma, O.; Santamaría, J.; Fadhel, M.A.; Al-Amidie, M.; Farhan, L. Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. J. Big Data
**2021**, 8, 53. [Google Scholar] [CrossRef] [PubMed] - Hecht-Nielsen, R. Counterpropagation networks. Appl. Opt.
**1987**, 26, 4979–4984. [Google Scholar] [CrossRef] - Kohonen, T. The self-organizing map. Neurocomputing
**1998**, 21, 1464–1480. [Google Scholar] [CrossRef] - Hojjat, A.; Park, H.S. Counterpropagation neural networks in structural engineering. J. Struct. Eng.
**1995**, 121, 1205–1212. [Google Scholar] - Yu, H.; Xie, T.; Hamilton, M.; Wilamowski, B. Comparison of different neural network architectures for digit image recognition. In Proceedings of the 2011 4th International Conference on Human System Interactions (HSI 2011), Yokohama, Japan, 19–21 May 2011. [Google Scholar]
- Wu, C.; Shivakumar, S. Back-propagation and counter-propagation neural networks for phylogenetic classification of ribosomal RNA sequences. Nucleic Acids Res.
**1994**, 22, 4291–4299. [Google Scholar] [CrossRef] [PubMed] - Kaveh, A.; Iranmanesh, A. Comparative Study of Backpropagation and Improved Counterpropagation Neural Nets in Structural Analysis and Optimization. Int. J. Space Struct.
**1998**, 13, 177–185. [Google Scholar] [CrossRef] - Juang, S.C.; Tarng, Y.S.; Lii, H.R. A comparison between the back-propagation and counter-propagation networks in the modeling of the TIG welding process. J. Mater. Process. Technol.
**1998**, 75, 54–62. [Google Scholar] [CrossRef] - Woods. Back and counter propagation aberrations. In Proceedings of the IEEE 1988 International Conference on Neural Networks, San Diego, CA, USA, 24–27 July 1988; Volume 1, pp. 473–479. [Google Scholar] [CrossRef]
- Hammer, B.; Micheli, A.; Sperduti, A.; Strickert, M. Recursive self-organizing network models. Neural Netw.
**2004**, 17, 1061–1085. [Google Scholar] [CrossRef] - Dittenbach, M.; Merkl, D.; Rauber, A. The growing hierarchical self-organizing map. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, Como, Italy, 27 July 2000; Volume 6. [Google Scholar]
- Saxena, A.; Goebel, K.; Simon, D.; Eklund, N. Damage Propagation Modeling for Aircraft Engine Run-to-Failure Simulation. In Proceedings of the 1st International Conference on Prognostics and Health Management (PHM08), Denver, CO, USA, 6–9 October 2008. [Google Scholar]
- Novic, M. Kohonen and counterpropagation neural networks applied for mapping and interpretation of IR spectra. In Artificial Neural Networks: Methods and Applications; Humana: Totowa, NJ, USA, 2009; pp. 43–58. [Google Scholar]
- Stojković, G.; Novič, M.; Kuzmanovski, I. Counter-propagation artificial neural networks as a tool for prediction of pKBH+ for series of amides. Chemom. Intell. Lab. Syst.
**2010**, 102, 123–129. [Google Scholar] [CrossRef] - Wu, C.; Chen, H.L.; Chen, S.C. Counter-Propagation Neural Networks for Molecular Sequence Classification: Supervised LVQ and Dynamic Node Allocation. Appl. Intell.
**1997**, 7, 27–38. [Google Scholar] [CrossRef] - Kohonen, T. Physiological interpretation of the self-organizing map algorithm. Neural Netw.
**1993**, 6, 895–905. [Google Scholar] [CrossRef] - Yin, H. The self-organizing maps: Background, theories, extensions and applications. In Computational Intelligence: A Compendium; Springer: Berlin/Heidelberg, Germany, 2008; pp. 715–762. [Google Scholar]
- Guimarães, G.; Lobo, V.S.; Moura-Pires, F. A taxonomy of Self-organizing Maps for temporal sequence processing. Intell. Data Anal.
**2003**, 7, 269–290. [Google Scholar] [CrossRef] - Villmann, T.; Bauer, H.-U. Applications of the growing self-organizing map. Neurocomputing
**1998**, 21, 91–100. [Google Scholar] [CrossRef] - Bucknam, J.S. Data Analysis and Processing Techniques for Remaining Useful Life Estimations. Master’s Thesis, Rowan University, Glassboro, NJ, USA, 2017. Available online: https://rdw.rowan.edu/etd/2430.
- Frasconi, P.; Gori, M.; Tesi, A. Successes and failures of backpropagation: A theoretical investigation. Prog. Neural Netw.
**1993**, 5, 1–38. [Google Scholar] - Hinton, G.E. Connectionist learning procedures. In Machine Learning; Morgan Kaufmann: Burlington, MA, USA, 1990; pp. 555–610. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst.
**2017**, 30, 1–11. [Google Scholar]

**Figure 2.**$MAE$ vs. number of epochs of the C-MPASS dataset. (

**a**) $MAE$ vs. number of epochs of FD001. (

**b**) $MAE$ vs. number of epochs of FD002. (

**c**) $MAE$ vs. number of epochs of FD003. (

**d**) $MAE$ vs. number of epochs of FD004.

**Figure 3.**Dendrogram of the C-MPASS dataset. (

**a**) Dendrogram of FD001. (

**b**) Dendrogram of FD002. (

**c**) Dendrogram of FD003. (

**d**) Dendrogram of FD004.

**Figure 4.**Prediction curves for RUL values of C-MPASS dataset. (

**a**) FD001. (

**b**) FD002. (

**c**) FD003. (

**d**) FD004.

Parameter | FD001 | FD002 | FD003 | FD004 |
---|---|---|---|---|

Number of training trajectories | 100 | 260 | 100 | 249 |

Number of testing trajectories | 100 | 259 | 100 | 248 |

Number of conditions | 1 | 6 | 1 | 6 |

Number of fault modes | 1 | 1 | 2 | 2 |

Ref. | Year | Architecture | FD001 | FD002 | FD003 | FD004 | ||||
---|---|---|---|---|---|---|---|---|---|---|

$\mathit{R}\mathit{M}\mathit{S}\mathit{E}$ | $\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}$ | $\mathit{R}\mathit{M}\mathit{S}\mathit{E}$ | $\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}$ | $\mathit{R}\mathit{M}\mathit{S}\mathit{E}$ | $\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}$ | $\mathit{R}\mathit{M}\mathit{S}\mathit{E}$ | $\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}$ | |||

[25] | 2016 | CNN | 18.44 | 1286.7 | 30.29 | 1375.0 | 19.81 | 159.62 | 20.15 | 788.64 |

[26] | 2018 | CNN | 12.61 | 273.7 | 22.36 | 10,412 | 12.64 | 284.1 | 23.31 | 12,466 |

[28] | 2020 | EWC | 12.56 | 231 | 22.73 | 3366 | 12.10 | 251 | 22.66 | 2840 |

[22] | 2022 | LSTM and MLP | 7.78 | 100 | 17.64 | 1440 | 8.3 | 104 | 17.63 | 2390 |

[23] | 2022 | LSTM | 11.35 | 213.65 | 17.78 | 1512.18 | 9.65 | 191.37 | 22.21 | 3285.51 |

[30] | 2023 | DRL | 12.17 | 208.06 | 16.28 | 1436.81 | 13.08 | 225.50 | 18.87 | 1725.74 |

[31] | 2023 | IGRU | 12.34 | 238 | 15.59 | 1205 | 13.12 | 292 | 13.25 | 1020 |

[32] | 2023 | CNN and GRU | 16.29 | 270.78 | 31.46 | 1014.90 | 23.71 | 583.14 | 41.13 | 1722.93 |

[33] | 2023 | LSTM and CNN | 3.52 | 29.98 | 13.29 | 693.46 | 4.44 | 32.96 | 13.79 | 720.64 |

[34] | 2023 | RGCN | 11.18 | 173.59 | 16.22 | 1148.16 | 11.52 | 225.03 | 19.11 | 2215.9 |

[35] | 2023 | STG | 11.62 | 203 | 13.04 | 738 | 11.52 | 198 | 13.62 | 816 |

[36] | 2022 | BiLSTMA | 13.78 | 255 | 15.94 | 1280 | 14.36 | 438 | 16.96 | 1650 |

[37] | 2022 | BLS and TCN | 12.08 | 243.0 | 16.87 | 1600 | 11.43 | 244 | 18.12 | 2090 |

PMTr | 2023 | CP and ReGHSOM | 1.87 | 16.1 | 8.51 | 521.01 | 2.47 | 12.24 | 8.14 | 624.87 |

PMTs | 2023 | CP and ReGHSOM | 1.57 | 15.5 | 8.24 | 522.31 | 2.35 | 12.50 | 8.78 | 622.45 |

Architecture | Number of Parameters | FLOP Count | Prediction Time (CPU sec) |
---|---|---|---|

CNN | 626,442 | 792,101 | 39.4 |

LSTM | 752,412 | 851,025 | 44.5 |

DRL | 1,154,201 | 781,241 | 53.8 |

CNN and GRU | 921,410 | 891,021 | 50.3 |

LSTM and CNN | 981,410 | 951,041 | 53.7 |

RGCN | 890,124 | 984,024 | 52.1 |

BiLSTMA | 1,012,410 | 892,120 | 52.9 |

Proposed (FD001) | 35,100 | 12,411 | 13 |

Proposed (FD002) | 78,510 | 34,152 | 25 |

Proposed (FD003) | 45,412 | 25,102 | 20 |

Proposed (FD004) | 98,012 | 40,241 | 29 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Baz, M.
A Novel RUL Prognosis Model Based on Counterpropagating Learning Approach. *Aerospace* **2023**, *10*, 972.
https://doi.org/10.3390/aerospace10110972

**AMA Style**

Baz M.
A Novel RUL Prognosis Model Based on Counterpropagating Learning Approach. *Aerospace*. 2023; 10(11):972.
https://doi.org/10.3390/aerospace10110972

**Chicago/Turabian Style**

Baz, Mohammed.
2023. "A Novel RUL Prognosis Model Based on Counterpropagating Learning Approach" *Aerospace* 10, no. 11: 972.
https://doi.org/10.3390/aerospace10110972