Next Article in Journal
A Blockchain Based Secure IoT System Using Device Identity Management
Next Article in Special Issue
K-Means Clustering and Bidirectional Long- and Short-Term Neural Networks for Predicting Performance Degradation Trends of Built-In Relays in Meters
Previous Article in Journal
LiDAR Intensity Completion: Fully Exploiting the Message from LiDAR Sensors
Previous Article in Special Issue
Use of DSLR and Sonic Cameras to Detect and Locate High-Voltage Corona Discharges
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Novel Image-Based Diagnosis Method Using Improved DCGAN for Rotating Machinery

1
Department of Electrical, Electronics and Computer Engineering, University of Ulsan, Ulsan 44610, Korea
2
PD Technology Cooperation, Ulsan 44610, Korea
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(19), 7534; https://doi.org/10.3390/s22197534
Submission received: 8 September 2022 / Revised: 28 September 2022 / Accepted: 1 October 2022 / Published: 4 October 2022
(This article belongs to the Special Issue Sensing Technologies for Fault Diagnostics and Prognosis)

Abstract

:
Rotating machinery plays an important role in industrial systems, and faults in the machinery may damage the system health. A novel image-based diagnosis method using improved deep convolutional generative adversarial networks (DCGAN) is proposed for the feature recognition and fault classification of rotating machinery. First, vibration signal data from the rotating machinery is transformed into time–frequency feature 2-D image data by a continuous wavelet transform and used for fault classification with the neural network method. The adaptive deep convolution neural network (ADCNN) is then combined with the generative adversarial networks (GANs) to improve the performance of the feature self-learning ability from input data. Compared with different fault diagnosis methods, the proposed method has better performance for image feature classification in rotating machinery.

1. Introduction

Rotating machinery creates power for autonomous systems and is widely applied in modern industry, such as energy systems, transportation, and engine systems. Rotating machinery works under complex environments for long periods of time, which may cause damage to machinery components [1]. Therefore, fault diagnosis technologies are required to analyze condition information and improve the operation reliability of rotating machinery.
Traditional methods, including wavelet decomposition and empirical mode decomposition, are widely applied in feature analysis for rotating machinery; however, there are some drawbacks that influence their performances [2].
Machine learning based methods are widely applied in the feature recognition of rotating machinery; for example, support vector machines (SVMs) use statistical learning theory and can achieve reasonable accuracy prediction for feature information; however, SVM has reduced performance in the case of processing large amounts of data with high noise information, and machine learning methods do not have such self-learning ability after presetting parameters [3,4,5].
Deep learning methods can achieve self-learning from training data and improve the prediction performance for nonlinear operation. Therefore, deep learning methods are one of the most popular technologies in data processing. For example, the convolutional neural network (CNN) method can change weights to adjust for its learning ability and achieves flexible structure for fault diagnosis [6,7,8,9]. To improve the self-learning ability of CNN, adaptive deep convolution neural network (ADCNN) methods have been applied to the training and testing of large data, identifying the target states for monitoring systems [10,11].
To improve the recognition performance with limited data, the generative adversarial network (GAN) method is applied to expand the original feature information and generative multiple synthetic information for neural networks. The process can develop different branches using a real sample to generate similar fake samples, which can improve the self-learning ability and classification recognition [12,13,14,15]. However, to maintain a balance between the two branch networks of the GAN architecture, the process may be difficult because the gradient disappears. As a result, the enhanced GAN was proposed to improve the synthesis quality [16,17]. A framework using deep convolutional generative adversarial networks (DCGAN) is proposed for solving model collapse and generating a high-quality process for data [18,19,20].
To further improve self-learning for training and testing, ADCNN is combined with DCGAN to develop a novel, multiple deep convolutional generative adversarial network (MDCGAN) for fault diagnosis in rotating machinery. In summary, the main contributions are as follows:
(1)
A novel image diagnosis method using improved DCGAN is proposed for the feature recognition and fault classification in rotating machinery to ensure the safety of the operation.
(2)
In the proposed method, vibration signal data from the rotating machinery is transformed into time–frequency feature 2-D images data by a continuous wavelet transform (CWT) with time–frequency characteristics for vibration signals. The vibration signal data are better for training and testing neural networks than 1-D dimensional signals.
(3)
ADCNN is combined with GANs that have two neural networks, a generative G and a discriminator D, to improve the self-learning ability of the architecture. An improved DCGAN architecture was developed by adding different layers to different parts of a generative G and a discriminator D.
(4)
Experiments using data from the Ulsan Industrial Artificial Intelligent (UIAI) Laboratory were used to verify the proposed method, which had better performance for image feature classification in rotating machinery than other fault diagnosis methods.
The rest of the paper is organized as follows. Section 2 and Section 3 provide a theoretical background. Detailed information on the proposed method is described in Section 4. The experimental result and discussion are shown in Section 5 and Section 6, respectively. Finally, Section 7 includes the conclusions.

2. Continuous Wavelet Transform

The original data from vibration signals is a 1-D signal, which only shows amplitude information and is not suitable for process by deep learning methods, so before training and testing, The vibration signal is preprocessed and representative features for deep learning methods are extracted to bear health state classification. A defect in the bearing produces shocks, resulting in complex variations in terms of amplitude and the distribution of impulses in the vibration signals. Utilizing the complex variation in the time series vibration signal allows a better understanding of the health state of the system under study, such as bearings. Entropy is one of the powerful metrics to evaluate these kinds of complexities in the signal. Landauskas et al. [8] constructed the pattern of permutation entropy as a function of time delay, which offers a way to unveil the presence of structures in the vibration signal due to bearing health conditions on multiple temporal scales in the form of 2D images. The method provides a precise description of complex nonstationary signals, however, from the literature, it is evident that the permutation entropy calculated as a function of the time delay does not take information regarding the amplitudes [21].
To utilize the variations in the vibration signals due to the bearing working conditions in this study, continuous wavelet transform with source wavelet Morse having the symmetry parameter = 3, is used. For complete details about the Morse wavelet, readers are advised to refer to [22]. The continuous wavelet transform (CWT) is used to transform vibration signals and obtain 2-D time-frequency images; 2-D images not only show the frequency energy information immediately, but also have some advantages for deep methods. The wavelet transform makes use of basic functions, and the variable window to remove noises effectively from raw signals with noises, so CWT is more suitable for nonstationary vibration signals. Meanwhile, the transform can extract important accurate information from the vibration signals and achieves feature localization in time-frequency energy, so the CWT method is more effectively than other signal analysis methods [4].
The continuous wavelet transform (CWT) method uses a wavelet function to analyze the vibration signals x ( t ) and ensures the correlation information that is used for analysis. The process is defined by the following formulate [4,7]:
C w ( a , b ) = + x ( t ) ψ a , b d t = 1 a   + x ( t ) ψ   ( t b ) a   d t
where ψ is the point continuous mother wavelet, a and b are the parameters of the wavelet function, the a denotes the scale and b makes sure the translation for basis wavelet, these parameters have continuous characteristics and used for wavelet transform. The CWT method also can show the energy information, ψ ( t ) denotes the complex conjugate from the wavelet basis and the 1 / a represents the energy-normalized coefficient, the finite amount energy of the wavelet function can be described as follow:
E = + | ψ | 2 d t <   C ψ = + | φ ( w ) | 2 w d w <
where the φ ( w ) is the Fourier transform and described as follow:
φ ( w ) = + ψ ( t ) e i w t d t
where w = 2 π f is the circular frequency, the CWT can show the energy information on the spectrum from vibration signals.

3. Introduction of Adaptive Deep Learning Neural Networks

ADCNN evolves from the basic theory of CNNs, mainly using convolution layers, pooling layers, several fully connected layers, and one final output layer, to achieve feature extraction from input data [11].

3.1. Convolution Layer

The convolution layers use kernels to adjust interactions and the convolutional operation, the convolutional process can filter redundancy information from the input data and extract feature information for the next layer [8]. The whole convolution process is shown in the following function:
  x n m = f ( i K n x i m 1 w i n m + b n m )
where x n m denotes the result of the mth kernel in the convolutional layer n, K n is the convolution region, w i n m is the weight in convolutional layer i n , b n m denotes the bias vector, and f ( ) is a nonlinear activation function.

3.2. Pooling Layer

After the convolutional layer with convolutional operation, the max pool layer uses the maximum value to reduce the dimensions and extract useful feature information from a previous output [5]. This architecture can also reduce the risk of overfitting. The whole process is described as follows:
x n m = f ( w n m max ( x n m 1 ) + b n m )
where max ( ) represents the max pooling function, w n m denotes the weight, x n m is the output, and b n m is the bias.

3.3. Full Connected Layer

At the end of CNN, a full connected layer is usually used to map feature information into the final classifier, the specific calculation process can be described as:
y z = f ( w z x z 1 + b z )
where y z is the output for the final connected layer and z is the network.

3.4. Backward Propagation

The loss function process is determined by the cross-entropy loss formula, and it is an important aspect of a neural network that makes it feasible for use as a gradient of the error function to achieve updated weights and biases for the prediction formular, and finally optimize the global target parameters from input data information [10]. The cross-entropy loss formula is expressed as:
E ( w ) = 1 n z = 1 n [ y z l n y z ¯ + ( 1 y z ) ln ( 1 y z ¯ ) ]
where y z ¯ represents the predicted value and y z is the actual target.

4. The Modified DCGAN

The DCGAN architecture is developed from the GAN and CNN. A traditional GAN mainly has two parts, a generator (G) and a discriminator (D), as shown in Figure 1. Generative G data from random noise, is then combined with the real data to be used in D; these models use minimax adversarial training to improve both networks’ circulation [15]. The object formular is described as follows:
max L D D = E x ~ P d a [ l o g D ( x ) ] + E z ~ p z [ log ( 1 D ( G ( z ) ) ) ]   minL G G = E z ~ p z [ log ( 1 D ( G ( z ) ) ) ]
where P d a is the x distribution, p z is the z distribution, D ( x ) is the possibility for real distribution, G ( z ) is the generated sample, and E is the expectation.
The DCGAN model combines GAN with a CNN method. GAN includes two neural networks: a generative G and a discriminator D. Compared with a traditional GAN method, the networks are conditioned on different layers in the DCGAN architecture [18]. To improve DCGAN, a modified version is proposed where the two neural networks are changed by different layers with different parts of generative G and discriminator D. The improved DCGAN is then used to classify the 2-D image datasets for vibration signals and achieve high accuracy classification. The architecture is described in Figure 2.
In this improved DCGAN method, some parameters are optimized to reduce the collapse of the network architecture, which can take advantage of the distribution convolutional layers to generate feature information for training and testing. Before training, vibration signals are transformed into 2-D time–frequency feature images by CWT, which provides an overcomplete representation of a signal by the translation and scale, the main function is described as C w ( a , b ) = + x ( t ) ψ a , b d t , where ψ is the continuous mother wavelet, a is the scale, and b is for translation. The same process was previously described. To make the proposed method stable for training image data, batch normalization (BN) and ReLU activation function were used to optimize the weight parameters for the gradient process.
In the improved DCGAN, the convolution kernel was optimized to select the size for feature information, the size of the convolution kernel and stride was 3 × 3, and the filter was 16. The dropout was 0.02 in the generative model. Some layers of the generative mode are shown in Figure 3 and the structure information is illustrated in Table 1.
In the discriminative model, the size of the convolution kernel and stride was 3 × 3, the filter was 16, and the dropout was 0.02, as shown in Figure 4. The structure information is described in Table 2.

5. Experimental Results

To verify the performance of the proposed method, the bearing tested platform was designed by the Ulsan Industrial Artificial Intelligent (UIAI) Laboratory of Ulsan University, South Korea [7]. As shown in Figure 5, the vibration bearing data includes four different conditions: inner race damaged bearings, outer race damaged bearings, roller race damaged bearings, and a normal bearing; the collected data can achieve classification for four different conditions by the proposed method. In this experiment, a speed of 1800 rpm was used for the three-phase motor and the sampling rate was 25 kHz for the vibration data.
FAG NJ206-3-TVP2 bearing is used during the experiment having a severe crack with a length 3 mm, a width of 0.3 mm, and a depth of 1mm on the outer race, roller, and inner race, respectively. The crack was created using an electronic machine called an electro-discharge machining; AE sensors (R151-AST type), and an accelerometer (PCB-622B01) were used to record vibration signals, and then a NI-9234 DAQ device was used to collect a large amount of data from the sensors. Table 3 provides details about the devices used in this experiment.
Figure 6 illustrates the frequency spectrum of the vibration signal obtained from the bearing under different operating conditions. The defect frequencies for inner race, outer race, and ball fault, are calculated using the formulation given in [23]. It can be seen from Figure 6a that when the bearing health condition changes from normal to defective condition, the fault frequency starts appearing in the vibration spectrum, as can be seen in Figure 6a–d. Thus, the collected vibration signals can be used for the fault diagnosis of the bearing. Therefore, the 1-D is further preprocessed and is transformed into 2-D time-frequency images by the CWT method; 2-D time-frequency images can show the time-frequency feature and energy feature, which are easier to identify. The improved DCGAN method was used for training and testing, then the classification results were calculated. The whole experiment process for the proposed method is described in Figure 7.
The transform results are described in Figure 8. The 1-D vibration signals can show the amplitude information, and these vibration signals also have some noises that influence feature recognition, with better analysis for the proposed method. The 1-D vibration signals were transformed into 2-D time-frequency images by the CWT method, and the 2-D time–frequency images are described in Figure 8B. These images can show the time frequency information that removes noise and shows the important energy information that is easier for localization, recognition, and classification. The input image size is 64 × 64. Then, the improved DCGAN was used to train and test the 2-D image data.
There are many redundancies in the 2-D image space; only energy features are used for classification. To describe the self-learning process of the improved DCGAN method, the convolutional process is used to extract features and reduce redundancy information. The convolutional process can filter noise signals and reduce high-dimensional features by each layer, and extraction features are easier with regards to recognition and classification.
After obtaining all 2-D image data by the CWT method, the 2-D images were processed by the improved DCGAN method. The proposed method makes use of the advantages of convolutional layers in the ADCNN method to extract the feature information and remove noise from the input 2-D images; the ADCNN method has a stronger self-learning ability that can filter redundancy information, so the feature information is easier to classify, and some processes showed results for the following conditions: normal, inner race fault, outer race fault, and roller race fault. The results for the convolutional process are described in Figure 9:
To describe the cluster ability process of the improved DCGAN method, the convolutional process can filter noise signals and reduce high-dimensional features by each layer, and the t-distributed stochastic neighbor embedding (t-SNE) algorithm is used to extract features and reduce redundancy information; it also shows a visualization with a two-dimensional plane.
To evaluate the performance of the proposed method a proper configuration of testing and training sets are made. As such, four different configurations of training and testing sets are prepared. The first configuration of the dataset contains 744 training samples and 496 testing samples, this configuration will be referred to as “A” in the text. A second configuration of the dataset is made, which contains 868 samples for training and 372 samples for testing, this configuration will be referred to as “B” in the text. For the third configuration, 992 samples were kept in the training set, while the remaining 248 samples were used for validation of the proposed model, this configuration will be referred to as “C” in the remaining parts of this study. For the fourth configuration, 1116 samples were used for training the proposed model and the remaining 124 samples were used for validating the proposed fault diagnosis method. Furthermore, in this study, t-distributed stochastic neighbor embedding (t-SNE) was used to show self-learning ability from the input 2-D image data.
After applying the proposed method to dataset configuration A, the method classified the bearing operating conditions with a total prediction accuracy of 99%. The discriminant feature space obtained from the proposed method for dataset configuration A is presented in Figure 10. From Figure 10a it can be observed that, initially, the t-SNE with a two-dimensional feature space does not show separability between the features. However, with different layers step by step, the samples representing different bearing operating conditions were gradually separated with high between class distance and less interclass scatteredness, as can be seen from Figure 10b–d. This is the reason for the higher classification accuracy of the improved DCGAN. Figure 11 shows the confusion matrix for the proposed method. From the confusion matrix per class, prediction accuracies were calculated and are presented in Table 4. From Table 4 it can be observed that: the method classified the inner race fault with the accuracy of 97%; outer race fault with 100% accuracy; roll race fault with 100% accuracy; and normal bearing operating condition with 99% accuracy. The reference method ADCNN classified the bearing health conditions with a total prediction accuracy of 98.5%, which is slightly lower than the proposed method. For the inner race fault, the ADCNN achieved 96% accuracy; 98% for the outer race fault; 100% for the roll race fault; and 99% for normal operating conditions. Another reference method used for comparison is DCNN. The method DCNN classified the bearing health conditions with a total prediction accuracy of 97.9%, which is lower than the proposed method. For the inner race fault, the DCNN achieved 94% accuracy; 99% for the outer race fault; 100% for the roll race fault; and 99% for normal operating conditions.
Continuously, the proposed method is applied to dataset configuration B and the performance was evaluated. From Table 4 it can be seen that the method classified the bearing operating conditions with a total prediction accuracy of 98.9%. The discriminant feature space obtained from the proposed method for dataset configuration B is presented in Figure 12. From Figure 12a it can be observed that, initially, the features have no discriminancy. However, as the sample goes through the layers of the improved DCGAN, the discriminancy of the features representing different bearing operating conditions improves, as can be seen from Figure 12b–d. Figure 13 shows the confusion matrix obtained for dataset configuration B. From Table 4 it can be observed that the method classified the inner race fault, outer race fault, roll race fault, and normal bearing operating condition with the accuracy of 97%, 99%, 100, and 99%, respectively. The reference method ADCNN classified the bearing health conditions with a total prediction accuracy of 98.3%, which is lower than the proposed method. For the inner race fault, outer race fault, roll race fault, and normal bearing operating condition, the ADCNN achieved 94%, 100%, 98%, 100, and 94 accuracies. For the reference method DCNN, a total prediction accuracy of 97.8% was obtained, which is lower than the proposed method. For the inner race fault, the DCNN achieved 91% accuracy, 99% for the outer race fault, 100% for the roll race fault, and 100% for normal operating conditions.
Similarly, the proposed method is applied to dataset configuration C. The method classified the bearing operating conditions with a total prediction accuracy of 98.9%. The discriminant feature space obtained from the proposed method for dataset configuration C is presented in Figure 14. From Figure 14a it can be observed that, initially, the t-SNE with a two-dimensional feature space does not show separability between the features. However, with different layers step by step, the samples representing different bearing operating conditions were gradually separated with high between class distance and less interclass scatteredness, as can be seen from Figure 14b–d. This is the reason for the higher classification accuracy of the improved DCGAN. Figure 15 shows the confusion matrix for the proposed method. From Table 4 it can be observed that the method classified the inner race fault with the accuracy of 100%, outer race fault with 95% accuracy, roll race fault with 100% accuracy, and normal bearing operating condition with 100% accuracy. The reference method ADCNN classified the bearing health conditions with a total prediction accuracy of 98%, which is lower than the proposed method. For the inner race fault, the ADCNN achieved 100% accuracy, 99% for the outer race fault, 99% for the roll race fault, and 100% for normal operating conditions. Another reference method used for comparison is DCNN. The method DCNN classified the bearing health conditions with a total prediction accuracy of 97.9%, which is lower than the proposed method. For the inner race fault, the DCNN achieved 94% accuracy, 100% for the outer race fault, 100% for the roll race fault, and 97% for normal operating conditions.
Continuously, the proposed method is applied to dataset configuration D and the performance was evaluated. From Table 4 it can be seen that the method classified the bearing operating conditions with a total prediction accuracy of 99.2%. The discriminant feature space obtained from the proposed method for dataset configuration D is presented in Figure 16. From Figure 16a it can be observed that, initially, the features have no discriminancy. However, as the sample goes through the layers of the improved DCGAN the discriminancy of the features representing different bearing operating conditions improves, as can be seen from Figure 16b–d. Figure 17 shows the confusion matrix obtained for dataset configuration D. From Table 4 it can be observed that the method classified the inner race fault, outer race fault, roll race fault, and normal bearing operating condition with the accuracy of 100%, 100%, 100%, and 97%, respectively. The reference method ADCNN classified the bearing health conditions with total prediction accuracy of 98.4%, which is lower than the proposed method. For the inner race fault, outer race fault, roll race fault, and normal bearing operating condition, the ADCNN achieved 94%, 100%, 100%, 100%, and 97% accuracies. For the reference method DCNN a total prediction accuracy of 97.8% was obtained, which is lower than the proposed method. For the inner race fault, the DCNN achieved 92% accuracy, 100% for the outer race fault, 100% for the roll race fault, and 98% for normal operating conditions. For a clearer understanding, the total prediction accuracies of the proposed and reference methods are presented in Figure 18.
To validate the sensitivity of the proposed method towards the incipient defects in the bearing, the Case Western University (CWU) bearing dataset is utilized. The CWU dataset and its configuration is used to validate the sensitivity of the proposed method towards incipient faults and can be explained as follows: the vibration signals were collected from the drive-end bearing with a sampling frequency of 12kHz under normal conditions. For outer race, ball, and inner race defect conditions, a crack of size 0.17 mm is created in the bearing and vibration signals with a sampling frequency of 12kHz were collected. During the experiment, the shaft speed was kept at 1797rpm. For a detailed description of the dataset, readers are advised to study [24]. Before applying the proposed method to the CWU dataset, a proper configuration of the dataset was made. As such, the data were divided into 80% of training data and the remaining 20% of the samples were used for validation purposes. After applying our proposed method to the CWU dataset we obtained 100% classification accuracy for normal conditions. The method classified the inner race fault, outer race fault, and ball defect with the accuracy of 97%, 99%, and 97%, respectively. The overall classification accuracy achieved by the proposed method for incipient bearing defects is 98.25%. The overall classification accuracy of the proposed method for incipient and severe faults is above 95%, which illustrates that the proposed method is sensitive to varying severity defects in the bearings, irrespective of the shaft speed.
The proposed method classified the bearing defects effectively under macrostructural vibration noise. The signal-to-noise ratio (SNR) for the inner race fault, ball fault, and outer race fault is calculated. The SNR for the inner race fault, ball fault, and outer race fault was −6.80, −5.74, and −3.84, respectively. From the SNR it can be concluded that the obtained vibration signals contain macrostructural vibration noise as the signal power is lower than the noise power. With such a low signal strength the proposed method classified the bearing health conditions with a prediction accuracy higher than the reference methods. The high classification accuracy of the proposed method shows that the proposed method is robust towards the bearing health condition classification irrespective of the vibration noise.
The GAN has two neural networks, a generative G and a discriminator D. Compared with traditional GAN, in the improved DCGAN architecture, two neural networks are changed by different layers through the different parts, which can improve the self-learning ability from input data. The improved DCGAN takes advantage of the convolutional layers and GAN to improve better performance than the DCNN method and ADCNN method for fault classification in rotating machinery.

6. Discussion

The evaluation parameters show that the improved DCGAN has a good performance for fault classification in rotating machinery. 1-D vibration signals were collected from the vibration bearing platform; these vibration signals only show amplitude information and some noises influence analysis for deep learning methods. To remove the noise and to show more information to the vibration signals, the CWT method was used to transform 1-D signals into 2-D images. These 2-D images can show the time-frequency feature and also localize some important energy information that are useful for recognition, however some pace information in these 2-D images is redundancy for training and testing, so improved DCGAN makes full use of the convolutional process, which can remove noise and reduce high dimensional information form input 2-D data. The convolutional process also shows the performance from input 2-D images. Finally, the extracted features were used for training and testing. To show the self-learning ability of classification, the t-NSE method is used to show the cluster performance for different layers. The input images are not separated into different conditions clearly, however with the different layer step, the different data conditions are classified into four conditions. Meanwhile, the improved DCGAN makes use of the convolutional layers and the GAN method to classify the fault 2-D images. GANs includes two neural networks, a generative G and discriminator D. In the improved DCGAN architecture, the two neural networks were changed by different layers via different generative G and a discriminator D. The improved DCGAN improves the performance of the self-learning ability from the input data, and the confusion matrix shows the prediction for the improved method. Compared with the DCNN and ADCNN methods, the improved DCGAN has high prediction accuracy. However, some methods for signal processing need more work in the future. As we know, the CWT method is a classical method that can transform 1-D signals into 2-D image, however, for CWT the selection of the source wavelet is very important, as the mother wavelet is the basis for decomposition. To choose the proper source wavelet, a lot of experiments, as well as expertise in signal processing, are needed. Therefore, for signal preprocessing, in the future, instead of CWT a self-adaptive signal processing method can be used, such as EMD, and EEMD.

7. Conclusions

In this paper, a novel fault classification method, based on improved DCGAN, is proposed for rotating machinery. For the first time, 1-D vibration signals were transformed into 2-D time-frequency feature images via the CWT method. The 2-D images showed the energy for different time–frequency, and the dataset information was more suitable for training and testing in the proposed method.
To further show the self-learning ability of different layers for these datasets, the t-SNE method was used to show the performance from raw datasets for final classification by different layers, demonstrating that the image data were well classified with the proposed method.
GANs includes two neural networks, a generative G and discriminator D. Compared with a traditional GAN, in the improved DCGAN architecture, the two neural networks were changed by different layers via different generative G and a discriminator D, which means ADCNN with GAN improved the performance of the self-learning ability from the input data.
Finally, the improved DCGAN takes advantage of the convolutional layers and GAN to achieve better performance than the DCNN and ADCNN methods.

Author Contributions

Conceptualization, Y.G., F.P. and J.-M.K.; methodology, Y.G., F.P. and J.-M.K.; validation, Y.G., F.P. and J.-M.K.; formal analysis, Y.G., F.P. and J.-M.K.; resources, Y.G., F.P. and J.-M.K.; writing—original draft preparation, Y.G. and F.P.; writing—review and editing, J.-M.K., visualization, Y.G. and F.P.; supervision, J.-M.K.; funding acquisition, J.-M.K. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Korea Technology and Information Promotion Agency(TIPA) grant funded by the Korea government(SMEs) (No. S3126818). This research was also funded by Ministry of Trade, Industry and Energy (MOTIE) and supported by Korea Evaluation Institute of Industrial Technology (KIET). [RS-2022-00142509, The development of simulation stage and digital twin for Land Based Test Site and hydrogen powered vessel with fuel cell] and by a National IT Industry Promotion Agency (NIPA) grant funded by the Korean government Ministry of Science and ICT (MSIT), Grant No. S1712-22-1001, for development of a smart mixed reality technology for improving the pipe installation and inspection processes in the offshore structure fabrication.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data will be provided upon request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sun, J.; Wang, L.; Li, J.; Li, F.; Li, J.; Lu, H. Online oil debris monitoring of rotating machinery: A detailed review of more than three decades. Mech. Syst. Signal Process. 2020, 149, 107341. [Google Scholar] [CrossRef]
  2. Hu, Q.; Si, X.-S.; Qin, A.-S.; Lv, Y.-R.; Zhang, Q.-H. Machinery Fault Diagnosis Scheme Using Redefined Dimensionless Indicators and mRMR Feature Selection. IEEE Access 2020, 8, 40313–40326. [Google Scholar] [CrossRef]
  3. Widodo, A.; Yang, B.S. Support vector machine in machine condition monitoring and fault diagnosis. Mech. Syst. Signal Process. 2007, 21, 2560–2574. [Google Scholar] [CrossRef]
  4. Neupane, D.; Kim, Y.; Seok, J. Bearing Fault Detection Using Scalogram and Switchable Normalization-Based CNN (SN-CNN). IEEE Access 2021, 9, 88151–88166. [Google Scholar] [CrossRef]
  5. Zhao, W.; Hua, C.; Dong, D.; Ouyang, H. A Novel Method for Identifying Crack and Shaft Misalignment Faults in Rotor Systems under Noisy Environments Based on CNN. Sensors 2019, 19, 5158. [Google Scholar] [CrossRef] [Green Version]
  6. Mukherjee, I.; Tallur, S. Light-Weight CNN Enabled Edge-Based Framework for Machine Health Diagnosis. IEEE Access 2021, 9, 84375–84386. [Google Scholar] [CrossRef]
  7. Toma, R.N.; Kim, C.-H.; Kim, J.-M. Bearing Fault Classification Using Ensemble Empirical Mode Decomposition and Convolutional Neural Network. Electronics 2021, 10, 1248. [Google Scholar] [CrossRef]
  8. Landauskas, M.; Cao, M.; Ragulskis, M. Permutation entropy-based 2D feature extraction for bearing fault diagnosis. Nonlinear Dyn. 2020, 102, 1717–1731. [Google Scholar] [CrossRef]
  9. He, J.; Wu, P.; Tong, Y.; Zhang, X.; Lei, M.; Gao, J. Bearing Fault Diagnosis via Improved One-Dimensional Multi-Scale Dilated CNN. Sensors 2021, 21, 7319. [Google Scholar] [CrossRef] [PubMed]
  10. Hasan, M.J.; Rai, A.; Ahmad, Z.; Kim, J.M. A Fault Diagnosis Framework for Centrifugal Pumps by Scalogram Based Imaging and Deep Learning. IEEE Access 2019, 9, 58052–58066. [Google Scholar] [CrossRef]
  11. Li, X.; Ye, M.; Liu, Y.; Zhu, C. Adaptive Deep Convolutional Neural Networks for Scene-Specific Object Detection. IEEE Trans. Circuits Syst. Video Technol. 2017, 29, 2538–2551. [Google Scholar] [CrossRef]
  12. Borji, A. Pros and cons of GAN evaluation measures: New developments. Comput. Vis. Image Underst. 2021, 215, 103329. [Google Scholar] [CrossRef]
  13. Hu, T.; Long, C.; Xiao, C. A Novel Visual Representation on Text Using Diverse Conditional GAN for Visual Recognition. IEEE Trans. Image Process. 2021, 30, 3499–3512. [Google Scholar] [CrossRef]
  14. Zou, L.; Li, Y.; Xu, F. An adversarial denoising convolutional neural network for fault diagnosis of rotating machinery under noisy environment and limited sample size case. Neurocomputing 2020, 407, 105–120. [Google Scholar] [CrossRef]
  15. Liu, S.; Jiang, H.; Wu, Z.; Li, X. Data synthesis using deep feature enhanced generative adversarial networks for rolling bearing imbalanced fault diagnosis. Mech. Syst. Signal Process. 2022, 163, 108139. [Google Scholar] [CrossRef]
  16. Jiang, Z.; Zhao, C.; Wang, H. Classification of Underwater Target Based on S-ResNet and Modified DCGAN Models. Sensors 2022, 22, 2293. [Google Scholar] [CrossRef] [PubMed]
  17. Liu, J.; Zhang, C.; Jiang, X. Imbalanced fault diagnosis of rolling bearing using improved MsR-GAN and feature enhancement-driven CapsNet. Mech. Syst. Signal Process. 2022, 168, 108664. [Google Scholar] [CrossRef]
  18. Fahimi, F.; Dosen, S.; Ang, K.K.; Mrachacz-Kersting, N.; Guan, C. Generative Adversarial Networks-Based Data Augmentation for Brain–Computer Interface. IEEE Trans. Neural Networks Learn. Syst. 2021, 32, 4039–4051. [Google Scholar] [CrossRef]
  19. Ma, L.; Li, Y.; Li, J.; Junior, J.M.; Goncalves, W.N.; Chapman, M.A. BoundaryNet: Extraction and Completion of Road Boundaries with Deep Learning Using Mobile Laser Scanning Point Clouds and Satellite Imagery. IEEE Trans. Intell. Transp. Syst. 2021, 23, 5638–5654. [Google Scholar] [CrossRef]
  20. Kalayeh, M.M.; Shah, M. Training Faster by Separating Modes of Variation in Batch-Normalized Models. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 42, 1483–1500. [Google Scholar] [CrossRef] [PubMed]
  21. Fadlallah, B.; Chen, B.; Keil, A.; Príncipe, J. Weightedpermutation entropy: A complexity measure for time series incorporating amplitude information. Phys. Rev. E 2013, 87, 022911. [Google Scholar] [CrossRef] [Green Version]
  22. Lilly, J.M. Element analysis: A wavelet-based method for analysing time-localized events in noisy time series. Proc. Math. Phys. Eng. Sci. 2017, 473, 20160776. [Google Scholar] [CrossRef] [Green Version]
  23. Maliuk, A.S.; Prosvirin, A.E.; Ahmad, Z.; Kim, C.H.; Kim, J.-M. Novel Bearing Fault Diagnosis Using Gaussian Mixture Model-Based Fault Band Selection. Sensors 2021, 21, 6579. [Google Scholar] [CrossRef]
  24. Ahmad, Z.; Hasan, M.J.; Kim, J.M. Transfer Learning with 2D Vibration Images for Fault Diagnosis of Bearings Under Variable Speed. In Proceedings of the International Conference on Intelligent Systems Design and Applications, Online, 13–15 December 2021; Springer: Cham, Switzerland, 2022. [Google Scholar]
Figure 1. The GAN structure.
Figure 1. The GAN structure.
Sensors 22 07534 g001
Figure 2. The architecture of the proposed method.
Figure 2. The architecture of the proposed method.
Sensors 22 07534 g002
Figure 3. Construction of the generative model.
Figure 3. Construction of the generative model.
Sensors 22 07534 g003
Figure 4. The architecture of the discriminative model.
Figure 4. The architecture of the discriminative model.
Sensors 22 07534 g004
Figure 5. Images of the bearing test platform.
Figure 5. Images of the bearing test platform.
Sensors 22 07534 g005aSensors 22 07534 g005b
Figure 6. Envelope–spectrum for different bearing operating condition: (a) normal, (b) inner race fault, (c) outer race fault, (d) ball race fault.
Figure 6. Envelope–spectrum for different bearing operating condition: (a) normal, (b) inner race fault, (c) outer race fault, (d) ball race fault.
Sensors 22 07534 g006
Figure 7. The whole experiment process for the proposed method.
Figure 7. The whole experiment process for the proposed method.
Sensors 22 07534 g007
Figure 8. 1−D vibration signals and 2−D time–frequency images for vibration signals: (a) normal, (b) inner race fault, (c) outer race fault, and (d) roller race fault.
Figure 8. 1−D vibration signals and 2−D time–frequency images for vibration signals: (a) normal, (b) inner race fault, (c) outer race fault, and (d) roller race fault.
Sensors 22 07534 g008aSensors 22 07534 g008b
Figure 9. The visualization for convolutional process in 2–D images under different condition. (a) 2–D images for normal, inner race fault, roller race fault, outer race fault vibration, (bd) visualization for convolutional process in (AD).
Figure 9. The visualization for convolutional process in 2–D images under different condition. (a) 2–D images for normal, inner race fault, roller race fault, outer race fault vibration, (bd) visualization for convolutional process in (AD).
Sensors 22 07534 g009aSensors 22 07534 g009b
Figure 10. The t-SNE for self-learning ability utilizing dataset configuration A: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Figure 10. The t-SNE for self-learning ability utilizing dataset configuration A: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Sensors 22 07534 g010
Figure 11. The confusion matrix for this classification.
Figure 11. The confusion matrix for this classification.
Sensors 22 07534 g011
Figure 12. The t-SNE for self-learning ability utilizing dataset configuration B: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Figure 12. The t-SNE for self-learning ability utilizing dataset configuration B: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Sensors 22 07534 g012
Figure 13. The confusion matrix for this classification.
Figure 13. The confusion matrix for this classification.
Sensors 22 07534 g013
Figure 14. The t-SNE for self-learning ability utilizing dataset configuration C: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Figure 14. The t-SNE for self-learning ability utilizing dataset configuration C: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Sensors 22 07534 g014
Figure 15. The confusion matrix for this classification.
Figure 15. The confusion matrix for this classification.
Sensors 22 07534 g015
Figure 16. The t-SNE for self-learning ability utilizing dataset configuration D: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Figure 16. The t-SNE for self-learning ability utilizing dataset configuration D: (a) raw data, (b) convolution layer, (c) convolution layer, and (d) final classification.
Sensors 22 07534 g016
Figure 17. The confusion matrix for this classification.
Figure 17. The confusion matrix for this classification.
Sensors 22 07534 g017
Figure 18. The performance of the different methods.
Figure 18. The performance of the different methods.
Sensors 22 07534 g018
Table 1. The parameter information for the generative model.
Table 1. The parameter information for the generative model.
No.Branch 1-Layer TypeKernel Size/Stride
1Convolution13*3
2Pooling12*2
3Dropout10.02
4Convolution23*3
5Pooling22*2
6Dropout20.02
7Flatten132
8Full-connectedlayer14
Table 2. The parameter information for the discriminative model.
Table 2. The parameter information for the discriminative model.
No.Branch 2-Layer TypeKernel Size/Stride
1Convolution13*3
2Pooling12*2
3Dropout10.02
4Convolution23*3
5Pooling22*2
7Flatten132
8Full-connectedlayer14
Table 3. Sensor information.
Table 3. Sensor information.
DevicesSpecification
RCB-622B01(Vibration sensor)Measurement range: ±490 m/s2
Frequency: 0.2–15,000 Hz
Sensor sensitivity: 100 mV/g
R151-AST (AE sensor)Operating range: 50–400 kHz
Resonant frequency:150 kHz (Ref in V/μbar)
Peak sensitivity: −22 dB (Ref in V/μbar)
DAQ-type NI 9234Dynamic range: 102 dB
Resolution: 24-bit
Operating temperature: −40 °C to 70 °C
Table 4. Prediction precision for different methods.
Table 4. Prediction precision for different methods.
Data Set
(Training/
Testing)
MethodsPrediction Accuracy (%)Total Prediction
Accuracy (%)
NormalInner Race
Fault
Outer
Race
Fault
Roll
Race
Fault
A (744/496)Proposed99%97%100%100%99.0%
ADCNN99%96%98%100%98.5%
DCNN99%94%99%100%97.9%
B
(868/372)
Proposed99%97%99%100%98.9%
ADCNN94%100%98%100%98.3%
DCNN100%91%99%100%97.8%
C
(992/248)
Proposed100%95%100%100%98.9%
ADCNN94%100%99%99%98.0%
DCNN97%94%100%100%97.9%
D
(1116/124)
Proposed97%100%100%100%99.2%
ADCNN97%94%100%100%98.4%
DCNN98%92%100%100%97.8%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gao, Y.; Piltan, F.; Kim, J.-M. A Novel Image-Based Diagnosis Method Using Improved DCGAN for Rotating Machinery. Sensors 2022, 22, 7534. https://doi.org/10.3390/s22197534

AMA Style

Gao Y, Piltan F, Kim J-M. A Novel Image-Based Diagnosis Method Using Improved DCGAN for Rotating Machinery. Sensors. 2022; 22(19):7534. https://doi.org/10.3390/s22197534

Chicago/Turabian Style

Gao, Yangde, Farzin Piltan, and Jong-Myon Kim. 2022. "A Novel Image-Based Diagnosis Method Using Improved DCGAN for Rotating Machinery" Sensors 22, no. 19: 7534. https://doi.org/10.3390/s22197534

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop