# Transformer Fault Diagnosis Method Based on TimesNet and Informer

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

_{2}), methane (CH

_{4}), acetylene (C

_{2}H

_{2}), ethane (C

_{2}H

_{4}), and ethylene (C

_{2}H

_{6}) five [16]. There are also some diagnostic methods in which carbon monoxide (CO) and carbon dioxide (CO

_{2}) need to be detected. There are also many transformer fault diagnosis methods proposed based on the DGA method, such as the Rogers ratio method [17] and the three-ratio method [18], in which the five selected characteristic gases constitute three pairs of ratios by calculating the values of C

_{2}H

_{2}/C

_{2}H

_{4}, CH

_{4}/H

_{2}, and C

_{2}H

_{4}/C

_{2}H

_{6}, corresponding to the different codes, which correspond to the different types of faults derived, respectively. However, the traditional DGA diagnostic method has outstanding defects in practical application; the internal faults of the transformer are very complex, and the coding combinations of the statistical analysis of typical accidents are too absolute and rough [19], which can diagnose very few faults, and if there is a fault that is not included in the coding combinations, it will lead to diagnostic inaccuracies or even misjudgment and omission of judgment.

- (1)
- A multilevel feature parallel extraction module based on the fusion of TimesNet and Informer is proposed for fault feature extraction in transformers.
- (2)
- Transformer DGA data are multi-periodic, and TimesNet is utilized to capture intra- and inter-periodic correlation properties of DGA data using 1D time series to 2D spatial transformations.
- (3)
- Incorporating the Informer model in TimesNet and utilizing the global correlation of the Informer Attention Mechanism module for better extraction of DGA data features.
- (4)
- Comparison experiments are designed to compare this paper’s method with classical models such as Transformer, Informer, TimesNet, etc., based on the public transformer DGA dataset to validate the effectiveness of the proposed method in transformer fault diagnosis. The experimental results show that the accuracy of the transformer fault diagnosis identification of the method proposed in this paper is higher than that of other models.

## 2. Materials and Methods

#### 2.1. TimesNet

#### 2.1.1. Timing Changes

#### 2.1.2. TimesBlock

- (1)
- One-dimensional tensor into two-dimensional tensor: extract the input one-dimensional temporal feature ${X}_{1D}^{l-1}$ cycles and transform it into a two-dimensional tensor to represent the two-dimensional temporal variation, which is given by:$${Y}^{l-1},\left\{{f}_{1},\cdots {f}_{n}\right\},\left\{{p}_{1},\cdots {p}_{n}\right\}=Period\left({X}_{1D}^{l-1}\right)$$$${X}_{2D}^{l,i}={Reshape}_{{p}_{i},{f}_{i}}\left(Padding\left({X}_{1D}^{l-1}\right)\right),i\in \left\{1,\cdots ,n\right\}$$
- (2)
- Extracting 2D time-varying features: for the 2D tensor $\left\{{X}_{2D}^{l,1},{X}_{2D}^{l,2},\cdots ,{X}_{2D}^{l,i}\right\}$, which has 2D localization, the information can be extracted using 2D convolution. Here, the original TimesBlock picks the classical Inception model:$${\widehat{X}}_{2D}^{l,i}=Inception\left({X}_{2D}^{l,i}\right)$$

- (3)
- Two-dimensional tensor into one-dimensional tensor: for the extracted temporal features, they are transformed back into a one-dimensional tensor to facilitate information aggregation with the formula:$${\widehat{X}}_{1D}^{l,i}=Teunc\left({Reshape}_{1,\left({p}_{i},{f}_{i}\right)}\left({\widehat{X}}_{2D}^{l,i}\right)\right),i\in \left\{1,\cdots ,n\right\}$$
- (4)
- Adaptive fusion: the obtained one-dimensional tensor $\left\{{\widehat{X}}^{l,1},\cdots ,{\widehat{X}}^{l,n}\right\}$ is weighted and summed with the intensity of its corresponding frequency to obtain the final output. Its formula is:$${\widehat{Y}}_{{f}_{1}}^{l-1},\cdots ,{\widehat{Y}}_{{f}_{n}}^{l-1}=Softmax\left({Y}_{{f}_{1}}^{l-1},\cdots ,{Y}_{{f}_{n}}^{l-1}\right)$$$${X}_{1D}^{l}=\sum _{i=1}^{n}{\widehat{Y}}_{{f}_{i}}^{l-1}\times {\widehat{X}}_{1D}^{l,i}$$

#### 2.2. Informer Model

#### 2.2.1. ProbSparse Self Attention

#### 2.2.2. Encoders

#### 2.2.3. Decoders

#### 2.3. Attention Mechanisms

#### 2.4. Convergence Model

## 3. Experiments and Analysis of Results

#### 3.1. Data Preparation

_{2}, CH

_{4}, C

_{2}H

_{6}, C

_{2}H

_{4}, and C

_{2}H

_{2}, were adopted as the criteria for fault diagnosis. The data were classified into six categories according to the type of faults [31], namely, high-energy discharge (HD), low-energy discharge (LD), high-temperature overheating (HT), low and medium temperature overheating (LT&MT), partial discharge (PD), and normal (NS) six classifications, and the data were divided into the training set and test set in the ratio of 8:2. The five gas concentrations and their state labels for each sample are shown in Figure 9. The details of the training and test set samples are shown in Table 1.

#### 3.2. Experimental Environment and Parameter Settings

#### 3.3. Ablation Experiments

#### 3.4. Comparative Experiments

#### 3.5. Experimental Results for Different Datasets

## 4. Conclusions

- (1)
- Introducing the Multidimensional Collaborative Attention Module MCA into the Inception structure of TimesBlock reduces the model complexity and the computational burden.
- (2)
- The MUSE attention mechanism in TimesNet was introduced to improve the feature extraction capability of the model for transformer faults.
- (3)
- The TimesNet and Informer multilayer parallel feature extraction modules are introduced in the construction of the feature module, making full use of the local features of the convolution and the global correlation of the attention mechanism module for feature aggregation, which can effectively extract the correlation characteristics of the time-series data within the cycle and between the cycles, as well as constructing the correlation information among the different local features for the long sequences, to allow the model to learn more time-series information to enhance its effectiveness in fault diagnosis.

- (1)
- All the different improvement methods positively affect the original TimesNet model, and the accuracy of the MCA-TimesNet model reaches 94.22%; the accuracy of the MUSE-TimesNet model reaches 94.85%; and the accuracy of the Hybrid model reaches 95.51%; the method in this paper combines the three improvement methods, and the final accuracy reaches 96.15%.
- (2)
- The fusion model of TimesNet and Informer proposed in this paper is more effective in transformer fault diagnosis and has a more extraordinary vital fault identification ability. The accuracy rate is improved by 3.21%, 4.48%, 4.07%, 8.33%, and 15.38%, respectively, compared with the single models of Informer, Autoformer, Transformer, DLinear, and MICN.
- (3)
- On different data sets, the fault recognition ability of the method proposed in this paper is also high, with an accuracy rate of 88.89%, which is highly applicable and can provide a reference for transformer fault diagnosis.

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Li, K.; Li, J.; Huang, Q.; Chen, Y. Data augmentation for fault diagnosis of oil-immersed power transformer. Energy Rep.
**2023**, 9, 1211–1219. [Google Scholar] [CrossRef] - Zheng, H.; Zhang, Y.; Liu, J.; Wei, H.; Zhao, J.; Liao, R. A novel model based on wavelet LS-SVM integrated improved PSO algorithm for forecasting of dissolved gas contents in power transformers. Electr. Power Syst. Res.
**2018**, 155, 196–205. [Google Scholar] [CrossRef] - Zhong, M.; Cao, Y.; He, G.; Feng, L.; Mo, Z.T.W.; Fan, J. Dissolved gas in transformer oil forecasting for transformer fault evaluation based on HATT-RLSTM. Electr. Power Syst. Res.
**2023**, 221, 109431. [Google Scholar] [CrossRef] - Qin, J.; Yang, D.; Wang, N.; Ni, X. Convolutional sparse filter with data and mechanism fusion: A few-shot fault diagnosis method for power transformer. Eng. Appl. Artif. Intell.
**2023**, 124, 106606. [Google Scholar] [CrossRef] - Hua, Y.; Sun, Y.; Xu, G.; Sun, S.; Wang, E.; Pang, Y. A fault diagnostic method for oil-immersed transformer based on multiple probabilistic output algorithms and improved DS evidence theory. Int. J. Electr. Power Energy Syst.
**2022**, 137, 107828. [Google Scholar] [CrossRef] - Ma, G.; Wang, Y.; Qin, W.; Zhou, H.; Yan, C.; Jiang, J.; Ju, Y. Optical sensors for power transformer monitoring: A review. High Voltage
**2020**, 6, 367–386. [Google Scholar] [CrossRef] - Zhao, B.; Yang, D.; Karimi, H.R.; Zhou, B.; Feng, S.; Li, G. Filter-wrapper combined feature selection and adaboost-weighted broad learning system for transformer fault diagnosis under imbalanced samples. Neurocomputing
**2023**, 560, 126803. [Google Scholar] [CrossRef] - Zou, D.; Xiang, Y.; Zhou, T.; Peng, Q.; Dai, W.; Hong, Z.; Shi, Y.; Wang, S.; Yin, J.; Quan, H. Outlier detection and data filling based on KNN and LOF for power transformer operation data classification. Energy Rep.
**2023**, 9, 698–711. [Google Scholar] [CrossRef] - Liu, X.; Xie, J.; Luo, Y.; Yang, D. A novel power transformer fault diagnosis method based on data augmentation for KPCA and deep residual network. Energy Rep.
**2023**, 9, 620–627. [Google Scholar] [CrossRef] - Xing, Z.; He, Y. Multi-modal information analysis for fault diagnosis with time-series data from power transformer. Int. J. Electr. Power Energy Syst.
**2023**, 144, 108567. [Google Scholar] [CrossRef] - Li, Z.; He, Y.; Xing, Z.; Chen, M. Minor fault diagnosis of transformer winding using polar plot based on frequency response analysis. Int. J. Electr. Power Energy Syst.
**2023**, 152, 109173. [Google Scholar] [CrossRef] - Tan, X.; Qi, J.; Gan, J.Q.; Zhang, J.; Guo, C.; Wan, F.; Wang, K. Multi-filter semi-supervised transformer model for fault diagnosis. Eng. Appl. Artif. Intell.
**2023**, 124, 106498. [Google Scholar] [CrossRef] - Zhong, M.; Yi, S.; Fan, J.; Zhang, Y.; He, G.; Cao, Y.; Feng, L.; Tan, Z.; Mo, W. Power transformer fault diagnosis based on a self-strengthening offline pre-training model. Eng. Appl. Artif. Intell.
**2023**, 126, 107142. [Google Scholar] [CrossRef] - Fan, Q.; Yu, F.; Xuan, M. Transformer fault diagnosis method based on improved whale optimization algorithm to optimize support vector machine. Energy Rep.
**2021**, 7, 856–866. [Google Scholar] [CrossRef] - Zhang, D.; Li, C.; Shahidehpour, M.; Wu, Q.; Zhou, B.; Zhang, C.; Huang, W. A bi-level machine learning method for fault diagnosis of oil-immersed transformers with feature explainability. Int. J. Electr. Power Energy Syst.
**2022**, 134, 107356. [Google Scholar] [CrossRef] - Hong, L.; Chen, Z.; Wang, Y.; Shahidehpour, M.; Wu, M. A novel SVM-based decision framework considering feature distribution for Power Transformer Fault Diagnosis. Energy Rep.
**2022**, 8, 9392–9401. [Google Scholar] [CrossRef] - Prasojo, R.A.; Putra, M.A.A.; Ekojono; Apriyani, M.E.; Rahmanto, A.N.; Ghoneim, S.S.M.; Mahmoud, K.; Lehtonen, M.I.; Darwish, M.M.F. Precise transformer fault diagnosis via random forest model enhanced by synthetic minority over-sampling technique. Electr. Power Syst. Res.
**2023**, 220, 109361. [Google Scholar] [CrossRef] - Xu, C.; Li, X.; Wang, Z.; Zhao, B.; Xie, J. Improved BLS based transformer fault diagnosis considering imbalanced samples. Energy Rep.
**2022**, 8, 1446–1453. [Google Scholar] [CrossRef] - Zhao, B.; Yang, M.; Diao, H.R.; An, B.; Zhao, Y.C.; Zhang, Y.M. A novel approach to transformer fault diagnosis using IDM and naive credal classifier. Int. J. Electr. Power Energy Syst.
**2019**, 105, 846–855. [Google Scholar] [CrossRef] - Benmahamed, Y.; Kherif, O.; Teguar, M.; Boubakeur, A.; Ghoneim, S.S.M. Accuracy Improvement of Transformer Faults Diagnostic Based on DGA Data Using SVM-BA Classifier. Energies
**2021**, 14, 2970. [Google Scholar] [CrossRef] - Zou, D.; Li, Z.; Quan, H.; Peng, Q.; Wang, S.; Hong, Z.; Dai, W.; Zhou, T.; Yin, J. Transformer fault classification for diagnosis based on DGA and deep belief network. Energy Rep.
**2023**, 9, 250–256. [Google Scholar] [CrossRef] - Peimankar, A.; Weddell, S.J.; Jalal, T.; Lapthorn, A.C. Evolutionary multi-objective fault diagnosis of power transformers. Swarm Evol. Comput.
**2017**, 36, 62–75. [Google Scholar] [CrossRef] - Tian, X.; Liu, Z.; Liu, J.; Shan, J.; Song, J.; Shu, H. Identification of overhead line fault traveling wave and interference clutter based on convolution neural network and random forest fusion. Energy Rep.
**2023**, 9, 1531–1545. [Google Scholar] [CrossRef] - Thomas, J.B.; Chaudhari, S.G.; Shihabudheen, K.V.; Verma, N.K. CNN-Based Transformer Model for Fault Detection in Power System Networks. IEEE Trans. Instrum. Meas.
**2023**, 72, 1–10. [Google Scholar] [CrossRef] - Han, X.; Huang, S.; Zhang, X.; Zhu, Y.; An, G.; Du, Z. A transformer condition recognition method based on dissolved gas analysis features selection and multiple models fusion. Eng. Appl. Artif. Intell.
**2023**, 123, 106518. [Google Scholar] [CrossRef] - Pérez-Chacón, R.; Asencio-Cortés, G.; Troncoso, A.; Martínez-Álvarez, F. Pattern sequence-based algorithm for multivariate big data time series forecasting: Application to electricity consumption. Future Gener. Comput. Syst.
**2024**, 154, 397–412. [Google Scholar] [CrossRef] - Zuo, C.; Wang, J.; Liu, M.; Deng, S.; Wang, Q. An Ensemble Framework for Short-Term Load Forecasting Based on TimesNet and TCN. Energies
**2023**, 16, 5330. [Google Scholar] [CrossRef] - Ren, S.; Wang, X.; Zhou, X.; Zhou, Y. A novel hybrid model for stock price forecasting integrating Encoder Forest and Informer. Expert Syst. Appl.
**2023**, 234, 121080. [Google Scholar] [CrossRef] - Yu, Y.; Zhang, Y.; Cheng, Z.; Song, Z.; Tang, C. MCA: Multidimensional collaborative attention in deep convolutional neural networks for image recognition. Eng. Appl. Artif. Intell.
**2023**, 126, 107079. [Google Scholar] [CrossRef] - Wei, H.; Wang, W.-S.; Kao, X.X. A novel approach to ultra-short-term wind power prediction based on feature engineering and informer. Energy Rep.
**2023**, 9, 1236–1250. [Google Scholar] [CrossRef] - Rao, S.; Zou, G.; Yang, S.; Barmada, S. A feature selection and ensemble learning based methodology for transformer fault diagnosis. Appl. Soft Comput.
**2024**, 150, 111072. [Google Scholar] [CrossRef]

**Figure 9.**Schematic of gas concentration and its labeling. (

**a**) Gas concentration; (

**b**) status labels.

**Figure 10.**Fault diagnosis results of each method. (

**a**) Methodological result of this paper; (

**b**) Informer result; (

**c**) Autoformer result; (

**d**) Transformer result; (

**e**) DLinear result; (

**f**) MICN result.

**Figure 11.**Confusion matrix for each method. (

**a**) Methodology of this paper; (

**b**) Informer; (

**c**) Autoformer; (

**d**) Transformer; (

**e**) DLinear; (

**f**) MICN.

Fault Type | Serial Number | Total Number of Samples | The Sample Size of the Training Set | The Sample Size of the Testing Set |
---|---|---|---|---|

high-energy discharge | HD | 45 | 36 | 9 |

low-energy discharge | LD | 45 | 36 | 9 |

high-temperature overheating | HT | 45 | 36 | 9 |

Low- and medium-temperature overheating | LT&MT | 45 | 36 | 9 |

partial discharge | PD | 45 | 36 | 9 |

normal | NS | 35 | 28 | 7 |

Models | Precision | Recall | Accuracy | F1 |
---|---|---|---|---|

TimesNet | 83.45% | 80.95% | 93.55% | 80.18% |

MCA-TimesNet | 84.15% | 83.3% | 94.22% | 81.72% |

MUSE-TimesNet | 86% | 85.2% | 94.85% | 84.18% |

Hybrid model | 87.9% | 87.05% | 95.51% | 86.18% |

Methodology of this paper | 89.52% | 88.9% | 96.15% | 88.4% |

Models | Precision | Recall | Accuracy | F1 |
---|---|---|---|---|

Methodology of this paper | 89.52% | 88.89% | 96.15% | 88.41% |

Informer | 84.49% | 77.52% | 92.95% | 77.63% |

Autoformer | 79.56% | 75.39% | 91.67% | 73.97% |

Transformer | 81.26% | 76.19% | 92.31% | 76.16% |

DLinear | 50.59% | 61.11% | 87.82% | 52.99% |

MICN | 40.82% | 41.27% | 80.77% | 33.96% |

Methodology of This Paper | Informer | Autoformer | Transformer | DLinear | MICN | |
---|---|---|---|---|---|---|

HD | 96.15% | 88.46% | 90.38% | 92.31% | 92.31% | 75% |

LD | 100% | 100% | 100% | 100% | 100% | 82.69% |

HT | 94.23% | 88.46% | 82.69% | 84.62% | 69.23% | 69.23% |

LT&MT | 92.31% | 90.38% | 84.62% | 88.46% | 82.69% | 82.69% |

PD | 96.15% | 98.07% | 96.15% | 96.15% | 96.15% | 88.46% |

NS | 98.07% | 92.31% | 96.15% | 92.31% | 86.53% | 86.53% |

average | 96.15% | 92.95% | 91.67% | 92.31% | 87.82% | 80.77% |

Fault Type | Serial Number | Total Number of Samples | The Sample Size of the Training Set | The Sample Size of the Testing Set |
---|---|---|---|---|

high-energy discharge | HD | 95 | 76 | 19 |

low-energy discharge | LD | 58 | 46 | 12 |

high-temperature overheating | HT | 109 | 87 | 22 |

low and medium-temperature overheating | LT&MT | 56 | 44 | 12 |

normal | NS | 37 | 30 | 7 |

Models | Precision | Recall | Accuracy | F1 |
---|---|---|---|---|

Methodology of this paper | 68.94% | 65.26% | 88.89% | 66.32% |

Informer | 61.93% | 58.24% | 86.11% | 58.3% |

Autoformer | 67.55% | 54.97% | 85.56% | 53.54% |

Transformer | 64.68% | 61.02% | 87.22% | 61.41% |

DLinear | 49.93% | 54.13% | 84.45% | 50.46% |

MICN | 56.81% | 50.88% | 83.89% | 50.34% |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Zhang, X.; Yang, K.; Zheng, L.
Transformer Fault Diagnosis Method Based on TimesNet and Informer. *Actuators* **2024**, *13*, 74.
https://doi.org/10.3390/act13020074

**AMA Style**

Zhang X, Yang K, Zheng L.
Transformer Fault Diagnosis Method Based on TimesNet and Informer. *Actuators*. 2024; 13(2):74.
https://doi.org/10.3390/act13020074

**Chicago/Turabian Style**

Zhang, Xin, Kaiyue Yang, and Liaomo Zheng.
2024. "Transformer Fault Diagnosis Method Based on TimesNet and Informer" *Actuators* 13, no. 2: 74.
https://doi.org/10.3390/act13020074