# One-Shot Fault Diagnosis of Wind Turbines Based on Meta-Analogical Momentum Contrast Learning

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Momentum Contrast Learning and Meta-Learning

#### 2.1. Momentum Contrast Learning

^{+}and x

^{−}are positive (similar) and negative (dissimilar) samples to x, respectively. f(•) is the encoder model for CL. score is a function to quantify similarity, e.g., cosine similarity.

^{+}and x

^{−}are positive (similar) and negative (dissimilar) samples to x, respectively.

_{0}, k

_{1}, k

_{2},…} that are the keys of a dictionary. Assume that there is a single key (denoted as k

^{+}) in the dictionary that q matches. With similarity measured by a dot product, a form of a contrastive loss function, called InfoNCE [33], is as follows:

_{q}and the momentum encoder as f

_{k}. Denote the parameter of f

_{k}as θ

_{k}and those of f

_{q}as θ

_{q}. Update the parameter θ

_{q}by back-propagation, and update the parameter θ

_{q}by:

#### 2.2. Meta-Learning-Based Few-Shot Learning

## 3. Meta-Analogical Momentum Contrast Learning

_{i}of its period reflects the fault characteristics as the fault characteristic frequency. When a bearing or gear fails, the period T of the shock pulse is determined by the area where the defect is generated; the inverse f

_{i}of its period reflects the fault characteristics as the fault characteristic frequency. When a machine is faulty, the spectral peaks of the fault characteristic frequency will appear in its vibration spectrum; thus, the fault characteristic frequency can be used as a feature for fault diagnosis. Spectrum analysis is the most useful fault diagnosis method. The fast Fourier transform converts complex time-domain signals into easy-to-analyze frequency-domain signals, which can more clearly represent the fault characteristic frequencies. After converting the data into spectral data by the fast Fourier transform, they can reflect the fault features more clearly and facilitate the neural network model to better learn and classify the features.

_{1}as 0.01. Backpropagation updates are carried out according to Equation (5).

_{2}as 0.002.

_{1}plays a more important role in updating the MA-MOCO model; the classification loss Loss

_{22}only plays a fine-tuning role. In Equation (8), Loss

_{1}is the contrast loss, and Loss

_{22}is the cross-entropy loss of the classification. The two losses jointly promote the model to achieve better optimization results and are not directly related. Because of the small sample size of the known fault data and the severe sample imbalance problem, the classification loss Loss

_{22}does not optimize the classification results well. Contrast loss Loss

_{1}is a loss function with the negative sample self-discovery property, which is essential for learning high-quality self-supervised representations and effectively learning knowledge common to the data. Therefore, this paper mainly relies on the contrast loss Loss

_{1}to update the model, and the classification loss Loss

_{22}as a supplement to further improve the model accuracy; thus, the coefficient γ is taken as 0.2.

Algorithm 1: One-Shot Fault Diagnosis Based on MA-MOCO. |

Input: Input training data $Tr={\left\{\left({x}_{i},{y}_{i}\right)\right\}}_{i=1}^{M}$, testing data $Te={\left\{{x}_{i}\right\}}_{i=1}^{I}$, classified model f_{ϕ}, encoder f_{q} and momentum encoder f_{k}, updated learning rate lr_{1} of baseline and lr_{2} of MA-MOCO, the model parameters θ, the momentum coefficient m, and the temperature hyper-parameter τ. |

########################(1) Pre-training baseline models #################### |

1: For each training epoch, do: |

2: For each batch, do: |

3: c_{i} = f(x_{i}) |

4: Backward propagation (with the learning rate as lr_{1}) by Equation (5). |

5: end |

########################(2) train MA-MOCO models ############################ |

6: Randomly draw data from Tr to form N tasks, each task containing k support sets and q query sets, to form $\{({S}_{1},{Q}_{1}),({S}_{2},{Q}_{2}),\cdots ,({S}_{n},{Q}_{n})\}$ |

7: For each training, do: |

8: For each batch, do: |

9: q_{i} = f_{q} (Q_{i}), k_{i} = f_{k} (S_{i}), c_{i} = Softmax(q_{i}) |

10: Update parameters of the encoder by: $Los{s}_{2}=-\mathrm{log}\frac{\mathrm{exp}({q}_{i}\cdot {k}_{i+}/\tau )}{{\displaystyle {\sum}_{i=0}^{K}\mathrm{exp}({q}_{i}\cdot {k}_{i}/\tau )}}+\gamma {\displaystyle \sum _{{x}_{i},{y}_{i}~{T}_{i}}[{y}_{i}\mathrm{log}{c}_{i}+(1-{y}_{i})\mathrm{log}(1-{c}_{i})]}$ |

11: Backward propagation of the encoder: ${\theta}_{q}\leftarrow {\theta}_{q}-l{r}_{2}{\nabla}_{\theta}Los{s}_{2}({\theta}_{q})$ |

12: Backward propagation of the momentum decoder: ${\theta}_{k}\leftarrow m{\theta}_{k}+(1-m){\theta}_{q}$ |

13: end |

###################### (3) testing results and t-SNE ######################### |

14: For the test set, calculate c_{Ti} = f (Te_{i}), calculate the accuracy, and draw the t-SNE diagram. |

Output: testing results. |

## 4. Case Analysis

#### 4.1. Case One: One-Shot Fault Diagnosis of the Generator Bearings for Wind Turbines

#### 4.2. Case Two: One-Shot Fault Diagnosis of the Wind Turbine Gearbox

#### 4.3. Discussion of the Results

## 5. Conclusions

## Author Contributions

## Funding

## Informed Consent Statement

## Conflicts of Interest

## References

- Wang, Y.; Yao, Q.; Kwok, J.T.; Lionel, M.N. Generalizing from a few examples: A survey on few-shot learning. ACM Comput. Surv.
**2020**, 53, 1–34. [Google Scholar] [CrossRef] - Hu, T.; Tang, T.; Lin, R.; Chen, M.; Han, S.; Wu, J. A simple data augmentation algorithm and a self-adaptive convolutional architecture for few-shot fault diagnosis under different working conditions. Measurement
**2020**, 156, 107539. [Google Scholar] [CrossRef] - Zheng, T.; Song, L.; Wang, J.; Teng, W.; Xu, X.; Ma, C. Data synthesis using dual discriminator conditional generative adversarial networks for imbalanced fault diagnosis of rolling bearings. Measurement
**2020**, 158, 107741. [Google Scholar] [CrossRef] - Ren, Z.; Zhu, Y.; Yan, K.; Chen, K.; Kang, W.; Yue, Y.; Gao, D. A novel model with the ability of few-shot learning and quick updating for intelligent fault diagnosis. Mech. Syst. Signal Process.
**2020**, 138, 106608. [Google Scholar] [CrossRef] - Schlegl, T.; Seebck, P.; Waldstein, S.M.; Ursula, S.; Langs, G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In Proceedings of the 2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; Lecture Notes in Computer Science. Volume 10265, pp. 146–157. [Google Scholar]
- Akcay, S.; Atapour-Abarghouei, A.; Breckon, T.P. GANomaly: Semi-supervised anomaly detection via adversarial training. In Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 July 2018; Lecture Notes in Computer Science. Volume 11363, pp. 622–637. [Google Scholar] [CrossRef][Green Version]
- Kingma, D.P.; Welling, M. Auto-encoding variational bayes. arXiv
**2014**, arXiv:1312.6114. [Google Scholar] - Makhzani, A.; Shlens, J.; Jaitly, N.; Goodfellow, L.; Frey, B. Adversarial autoencoders. arXiv
**2016**, arXiv:1511.05644. [Google Scholar] - Hospedales, T.M.; Antoniou, A.; Micaelli, P.; Storkey, A. Meta-learning in neural networks: A survey. IEEE Trans. Pattern Anal. Mach. Intell.
**2021**, 3079209. [Google Scholar] [CrossRef] - Wu, J.; Zhao, Z.; Sun, C.; Yan, R.; Chen, X. Few-shot transfer learning for intelligent fault diagnosis of machine. Measurement
**2020**, 166, 108202. [Google Scholar] [CrossRef] - Wang, D.; Zhang, M.; Xu, Y.; Lu, W.; Yang, J.; Zhang, T. Metric-based meta-learning model for few-shot fault diagnosis under multiple limited data conditions. Mech. Syst. Signal Process.
**2021**, 155, 107510. [Google Scholar] [CrossRef] - Feng, Y.; Chen, J.; Zhang, T.; He, S.; Xu, E.; Zhou, Z. Semi-supervised meta-learning networks with squeeze-and-excitation attention for few-shot fault diagnosis. ISA Trans.
**2021**, 120, 383–401. [Google Scholar] [CrossRef] - Su, H.; Xiang, L.; Hu, A.; Xu, Y.; Yang, X. A novel method based on meta-learning for bearing fault diagnosis with small sample learning under different working conditions. Mech. Syst. Signal Process.
**2022**, 169, 108765. [Google Scholar] [CrossRef] - Wang, S.; Wang, D.; Kong, D.; Wang, J.; Li, W.; Zhou, S. Few-shot rolling bearing fault diagnosis with metric-based meta learning. Sensors
**2020**, 20, 6437. [Google Scholar] [CrossRef] [PubMed] - Koch, G.; Zemel, R.; Salakhutdinov, R. Siamese neural networks for one-shot image recognition. In Proceedings of the 2015 International Conference on Machine Learning, Lille, France, 6–11 July 2015; Volume 1, pp. 1–8. [Google Scholar]
- Zhang, A.; Li, S.; Cui, Y.; Yang, W.; Dong, R.; Hu, J. Limited data rolling bearing fault diagnosis with few-shot learning. IEEE Access
**2019**, 7, 110895–110904. [Google Scholar] [CrossRef] - Cai, Q.; Pan, Y.; Yao, T.; Yan, C.; Mei, T. Memory matching networks for one-shot image recognition. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 4080–4088. [Google Scholar] [CrossRef][Green Version]
- Tran, K.; Sato, H.; Kubo, M. Memory Augmented Matching Networks for Few-Shot Learnings. Int. J. Mach. Learn. Comput.
**2019**, 9, 743–748. [Google Scholar] [CrossRef] - Sung, F.; Yang, Y.; Zhang, L.; Xiang, T.; Philip, H.S.T.; Timothy, M.H. Learning to compare: Relation network for few-shot learning. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 1199–1208. [Google Scholar] [CrossRef]
- Garcia, V.; Bruna, J. Few-shot learning with graph neural networks. In Proceedings of the 2017 International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; Volume 1, pp. 1–13. [Google Scholar] [CrossRef]
- Tian, Y.; Krishnan, D.; Isola, P. Contrastive Multiview Coding. In European Conference on Computer Vision; Springer: Cham, Switzerland, 2020; Lecture Notes in Computer Science; Volume 12356, pp. 776–794. [Google Scholar]
- Yang, J.; Chen, H.; Yan, J.; Chen, X.; Yao, J. Towards better understanding and better generalization of few-that classification in histology images with contrastive learning. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 19–24 June 2022. [Google Scholar] [CrossRef]
- Majumder, O.; Ravichandran, A.; Maji, S.; Polito, M.; Soatto, S. Revisiting Contrastive Learning for Few-Shot Classification. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021. [Google Scholar] [CrossRef]
- Liu, C.; Fu, Y.; Xu, C.; Yang, S.; Li, J.; Wang, C.; Zhang, L. Learning a Few-shot Embedding Model with Contrastive Learning. Natl. Conf. Assoc. Adv. Artif. Intell.
**2021**, 35, 8635–8643. [Google Scholar] - Kim, Y.; Shin, J.; Yang, E.; Hwang, S.J. Few-shot Visual Reasoning with Meta-analogical Contrastive Learning. Adv. Neural Inf. Processing Syst.
**2020**, 33, 16846–16856. [Google Scholar] - Liu, H.; Zhang, F.; Zhang, X.; Zhan, S.; Zhang, X. An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling]. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 1945–1955. [Google Scholar] [CrossRef]
- Chen, X.; Yao, L.; Zhou, T.; Dong, J.; Zhang, Y. Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Images. Pattern Recognit.
**2021**, 113, 107826. [Google Scholar] [CrossRef] - Ding, Y.; Zhuang, J.; Ding, P.; Jia, M. Self-supervised pretraining via contrast learning for intelligent incipient fault detection of bearings. Reliab. Eng. Syst. Saf.
**2022**, 218, 108126. [Google Scholar] [CrossRef] - Jaiswal, A.; Babu, A.R.; Zadeh, M.Z.; Banerjee, D.; Makedon, F. A survey on contrastive self-supervised learning. Technologies
**2021**, 9, 2. [Google Scholar] [CrossRef] - Gutmann, M.; Hyvärinen, A. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. J. Mach. Learn. Res.
**2010**, 9, 297–304. [Google Scholar] - He, K.; Fan, H.; Wu, Y.; Xie, S.; Girshick, R. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 9729–9738. [Google Scholar] [CrossRef]
- Chen, T.; Kornblith, S.; Norouzi, M.; Hinton, G. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 2020 International Conference on Machine Learning, Virtual, 12–18 July 2020; pp. 1597–1607. [Google Scholar]
- Oord, A.; Li, Y.; Vinyals, O. Representation Learning with Contrastive Predictive Coding. arXiv
**2018**, arXiv:1807.037482018. [Google Scholar] - Finn, C.; Abbeel, P.; Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 2017 International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; PMLR 70. pp. 1126–1135. [Google Scholar]

**Figure 3.**A 4-way 1-shot diagnosis of the generator bearing data when the training set of each class has only 5 fault samples.

**Figure 4.**A 4-way 1-shot diagnosis of the generator bearing data when the training set of each class has only 10 fault samples.

**Figure 5.**A 4-way 1-shot diagnosis of the generator bearing data when the training set of each class has only 20 fault samples.

**Figure 6.**A 4-way 1-shot diagnosis of the accuracy of the generator bearing data when the training set of each class has 5, 10, and 20 fault samples.

**Figure 7.**A 5-way 1-shot diagnosis of the generator bearing data when the training set of each class has only 5 fault samples.

**Figure 8.**A 5-way 1-shot diagnosis of the generator bearing data when the training set of each class has only 10 fault samples.

**Figure 9.**A 5-way 1-shot diagnosis of the generator bearing data when the training set of each class has only 20 fault samples.

**Figure 10.**A 5-way 1-shot diagnosis of the accuracy of the wind turbine gearboxes data when the training set of each class has 5, 10, and 20 fault samples.

Fault Type | Label | Number of Samples from the Training Set | Number of Samples from the Testing Set | |
---|---|---|---|---|

Healthy | No faults | 0 | 100 | 240 |

Fault 1 | Outer ring failure | 1 | 5 or 10 or 20 | 240 |

Fault 2 | Inner ring failure + Outer ring failure | 2 | 5 or 10 or 20 | 240 |

Fault 3 | Inner ring failure + Rolling failure + Cage failure | 3 | 5 or 10 or 20 | 240 |

Fault Type | Label | Number of Samples from the Training Set | Number of Samples from the Testing Set | |
---|---|---|---|---|

Healthy | No faults | 0 | 100 | 240 |

Fault 1 | Spalling of the gears in the intermediate shaft | 1 | 5 or 10 or 20 | 240 |

Fault 2 | Broken teeth of the gears in the intermediate and high-speed shaft | 2 | 5 or 10 or 20 | 240 |

Fault 3 | Broken teeth of the gears in the high-speed shaft | 3 | 5 or 10 or 20 | 240 |

Fault 4 | Broken teeth of the gears in the intermediate shaft | 4 | 5 or 10 or 20 | 240 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Liu, X.; Guo, H.; Liu, Y.
One-Shot Fault Diagnosis of Wind Turbines Based on Meta-Analogical Momentum Contrast Learning. *Energies* **2022**, *15*, 3133.
https://doi.org/10.3390/en15093133

**AMA Style**

Liu X, Guo H, Liu Y.
One-Shot Fault Diagnosis of Wind Turbines Based on Meta-Analogical Momentum Contrast Learning. *Energies*. 2022; 15(9):3133.
https://doi.org/10.3390/en15093133

**Chicago/Turabian Style**

Liu, Xiaobo, Hantao Guo, and Yibing Liu.
2022. "One-Shot Fault Diagnosis of Wind Turbines Based on Meta-Analogical Momentum Contrast Learning" *Energies* 15, no. 9: 3133.
https://doi.org/10.3390/en15093133