Transfer Learning-Based Intelligent Fault Detection Approach for the Industrial Robotic System

Abstract

With increasing customer demand, industry 4.0 gained a lot of interest, which is based on smart factories. In smart factories, robotic components are vulnerable to failure due to various industrial operations such as assembly, manufacturing, and product handling. Timely fault detection and diagnosis (FDD) is important to keep the industrial operation smooth. Previously, only the unloaded-based FDD algorithms were considered for the industrial robotic system. In the industrial environment, the robot is working under various working conditions such as speeds, loads, and motions. Hence, to reduce the domain discrepancy between the lab scale and the real working environment, we conducted experimentations under various working conditions. For that purpose, an extensive experimental setup is prepared to perform a series of various experiments mimicking the real environmental condition. In addition, in previous research work, various machine learning (ML) and deep learning (DL) approaches were proposed for robotic arm component fault detection. However, various issues are related to the DL and ML approaches. The ML models are problem-specific, and complex in computations. The DL model needs a huge amount of data. The DL model is composed of various layers that have not been thoroughly explored; as a result, the fault detection model lacks a comprehensive explanation. To overcome these issues, the transfer learning (TL) model is considered with the diverse experimental scenarios. The main contribution is to increase the generalization capabilities of the robotic PHM in the context of previously available research work. For that purpose, the VGG16 model is used because of its autonomous feature extractions for fault classification. The data are collected under a variety of different operating conditions such as loadings, speeds, and motion patterns. The 1D signal is converted to a 2D signal (scalogram) to perform the TL model. The proposed approach shows effective fault detection performance and has the capabilities of generalization under variable working conditions.

1. Introduction

The smart factory is evolving regularly because of high flexibility, deep integration, dynamic reconfiguration, and massive volume of data. The smart factory is composed of various advancements such as artificial intelligence (AI), big data analysis, Internet of Things (IoT), industrial internet, and cloud computing [1,2]. One of the important factors of smart factories is to boost productivity. Hence, it is mandatory to keep the industrial operation smooth without the downfall of each component. Prognostic health management (PHM) can be used to keep industrial operations running smoothly and consistently. In PHM, data-based techniques are prominent because of the issues related to the physics-based modeling [3].

Machine learning (ML) and deep learning (DL) algorithms have been utilized for the PHM of different mechanical systems [2,4,5]. These techniques are composed of supervised and unsupervised approaches. In supervised approaches, the labeled data are utilized for fault classification; however, in the case of unsupervised approaches, the unlabeled data can be used for fault detection [6,7,8]. According to ML approaches, handcrafted features are extracted, and the feature space is then reduced by selecting the most prominent feature using feature selection methods. Afterward, the selected feature can be used for fault classification using various ML classifiers. For instance, a feature selection approach with Gaussian Ant Lion optimizer (GALO) is integrated with the K-nearest neighbor (KNN) for the fault detection and diagnosis (FDD) of rotating machines [9]. Lee et al. [10] presented an FDD technique based on the feature variable dimensional coordination to reduce the computational cost. Buchaiah et al. [11] studied the bearing FDD using an ML approach based on the Bhattacharyya distance and SVM. Guo et al. [12] presented motor current signature analysis (MCSA) for the FDD using the multi-sensor data, the improved cyclic spectral covariance matrix (ICSCM), and the MCSA combined which completely sustained the connectivity of the various sensors. The DL model has been used for the FDD because of its robustness and ease in computational complexities as compared to the hand-crafted features. Surendran et al. [13] proposed a DL model using a sailfish optimizer (SFO) to optimize the hyperparameters for accurate FDD. Ma et al. [14] proposed an ensemble DL-based FDD for the bearing system to overcome the generalization performance by integrating different DL models with multi-objective optimizations. In addition, various DL models have been proposed for the FDD of rotating machinery [15,16,17].

The robotic system, especially the six degree of freedom (DOF) robot, is the backbone of the smart factory; hence, its PHM is mandatory to keep the industrial operations. Over the years, various approaches have been used for robotic PHM; ferrography analysis (FA) [18], vibration analysis (VA) [19], and acoustic emission analysis (AEA) [20,21]. However, various issues are concerned related those conventional approaches, such as installation of extra sensors, real-time fault detection, bulky working environment, implementation issues, and financial expenditures, etc. To overcome these issues, the encapsulated system of the robot motor current signature analysis (MCSA) is applied for the PHM analysis [22]. For instance, a feature engineering-based ML-model is proposed for the FDD of the robotic rotate vector (RV) based on simple motion [23]. In addition, a discrete wavelet transform (DWT) is used to analyze the electrical current data and infuse feature extraction and selection for the fault classification based on ML classifier. The handcrafted features, however, are inherently problematic, and the model lacks generalization capabilities. To overcome the issue, a DL model is developed for the robotic FDD. In the proposed approach, a deep wavelet scattering (DWS) is applied for unloaded robotic strain wave gear (SWG) reducer using two different kind of motion with variable speed of operation [24]. The proposed approaches show good performance for unloaded robotic reducer. However, in the proposed algorithms only unloaded conditions are considered. In real industrial scenarios, the robot is operating at various working conditions such as speeds, loads and motions. Hence, it is needed to reduce the domain discrepancy between the lab scale and actual situations. The extensive experimental setup is required to perform a series of various experiments mimicking the real environmental condition and various loading conditions. In addition, various issues are related to the DL approaches; the DL model is made up of various layers which are not well explored, hence, the FDD model lacks comprehensive explanation. The number of parameters and hyperparameters in deeper networks is enormous, necessitating a large amount of labeled data, and computational complexities [25]. To overcome the issues related to the DL and ML, the transfer learning (TL) model can be used with less computation and generalization capabilities [26,27,28,29,30]. Hence, in the proposed work, the TL-based component-level FDD is considered for the actual industrial robotic system which is operating at the real working conditions, such as loading, speed and various profiles of motions.

In the current study, TL model is proposed for the robotic PHM to ensure the generalization capabilities of the model. The extensive experimental setup is prepared to perform a series of various experiments mimicking the real environmental condition, and various loading conditions are considered. The main contribution was to increase the generalization capabilities of the robotic PHM in the context of previously available research work, as given above. For that purpose, the electrical current data of the robotic system is collected under real environmental conditions such as various loadings, speeds, and motion patterns. In the proposed two different motions such as simple and welding motion are considered with variable speed of operation by considering various loading conditions. The data segmentation is performed based on a single cycle of the robotic arm. The 1D signal is converted to a 2D signal to perform the TL model. The TL model is used to overcome data-related issues and data complexities. The TL-based VGG16 model is utilized for the first time in robotic PHM under real environmental conditions. The proposed approach is used for efficient features extraction for the FDD of robotic system, that eliminates the requirement to train the DL from scratch, allowing it to converge faster. The authenticity of the model is evaluated by different FDD parameters and shows excellent results for the FDD of the industrial robotic system.

The manuscript is arranged into different sections. Section 2 is composed of the overall methodology, experimental details, data acquisition, data preprocessing, and TL model. Section 3 overviews the results and discussions. Section 4 summarizes the conclusions of the proposed research work.

2. Methodology

The proposed methodology of the research work carried out is shown in Figure 1. In this work, the FDD model for the real industrial robot (Robostar RA004) is presented. The servomotor bearing fault of axis 3 of the robotic arm is considered. The data are collected under distinct operating conditions such as speed, motion, and loading conditions. To mimic the real working conditions, only the loading-based data are collected for the effective FDD model. The data preprocessing is carried out by data segmentation and synchronization. The 1D data are converted into 2D scalogram images for the FDD model. The TL model is used to overcome data-related issues and data complexities. The VGG16-based TL model is considered for the extraction of efficient features for the FDD of the robotic system. The authenticity of the VGG16 is based on the diverse type of data with variable working conditions. For instance, overall, image-based data for various loading conditions such as 500 g, 1000 g, 1500 g, 2000 g, 2500 g, and 3000 g are considered for training the model based on the simple and welding motion, separately. Hence, for both the motions, around 80% of the data is used for the training purpose. To check the generalization capabilities of the model the highest loading conditions are used for testing and validating the model. For that purpose, 20% of the data is used for testing and validation, considering 10% for testing and 10% for validation. Hence, the health status of the bearing is predicted for the highest loading condition (3500 g) using the trained model with the lower loading condition. TL eliminates the requirement to train the DL from scratch, allowing it to converge faster. Detailed methodology is presented in the upcoming sections.

2.1. Experimental Detials

In the proposed work, the industrial robot (Robostar RA004) is under consideration. It is a 6-DOF robot, where all six axes of the robot can move independently. Each axis of the robot is powered by servomotor to operate various industrial operations and electric motor for various motion of the axes. The specification of RA004 with related information of each joint is shown in

Table 1. Specifications of Robostar RA004.

Variable		Quantity (Unit)
Allowable distance		610 mm
Maximum load		4 kg
Repeated placement accuracy		±0.02 mm
Motion Range/Maximum Speed	J1	±170°/410°/s
	J2	−90° to +45°/410°/s
	J3	−210° to +61°/520°/s
	J4	±190°/560°/s
	J5	±130°/560°/s
	J6	±360°/900°/s

. Various parameters are mentioned such as maximum distance, payload, repetitive positioning accuracy, motion range and speed range. Two health states (healthy and faulty) are considered to carry the PHM process for the servomotor of the robotics arm. The robotic servomotor bearing PHM is considered with two health states (healthy and faulty). The inner bearing fault is induced in the inner race of the bearing at axis 3 of the robot. Figure 2 represents the flow of the experimental system used in the proposed research work.

2.2. Data Acquisition

The data acquisition (DAQ) system is illustrated in Figure 3. The robotic arm is powered by an electric motor and the electrical current data are collected using the current sensors (JS16FL-100). The sensors are connected to the robot power supply to observe the electrical current data. The DAQ system collects the data from the sensors and then the data are forwarded to the to the lab view personal computer (PC). The information on the collected is given in

Table 2. Experimental descriptions.

Variable	Description
Data type	1st phase of electrical current
Motion	Simple and welding motion
Speed	10~100% with 10% increment
Fault type	Inner bearing fault of servomotor
Fault location	Robotic 3rd axis
Health states	Normal, faulty
Sampling frequency	5 kHz
Loading conditions	500 g, 1000 g, 1500 g, 2000 g, 2500 g, 3000 g, and 3500 g

. In the experimental setup, two distinguished kinds of motion such as simple and welding motion are considered. In simple motion, the axis is simply moving forward and backward. On the other hand, all the axes are moving independently in the welding motion [22]. The electrical current data are collected for the robotic arm with 10 different speeds, such as 10% to 100% with an increment of 10%. To consider the actual environmental conditions, the data are collected under different loading conditions such as (500 g, 1000 g, 1500 g, 2000 g, 2500 g, 3000 g and 3500 g).

2.3. Data Preprocessing

Data preprocessing is very crucial to clean and organize the PHM process. In the proposed work, the data preprocessing is done in two steps. Initially, all the raw data are converted to segmentation based on per cycle and then 1D signals are converted into 2D images for the TL model for the robotic PHM. The upcoming sections describe the details of the data segmentation and data conversion.

2.3.1. Data Segmentation

The collected data from the current sensors are raw and cannot be used directly for the PHM process of the robotics system. The data needs to be cleaned by removing unnecessary information. In addition, the data for the same operating conditions with different health states needs to be synchronized to reduce the probability of confusion in the fault signature because of the data pattern rather than the health states. Pre-processing of signals is required for real-time FDD. It is hard to use continuously acquired signals for PHM, hence, pre-processing is required to divide the data into a single cycle from the raw data set to analyze the failure of a single cycle. The signal segmentation is carried out by using the following steps.

I.

Set a sample cycle from the overall raw signal of the collected data.

II.

Perform short-time Fourier transform (STFT) on the original signal and sample data.

III.

Based on the power spectral density (PSD) of the sample data from STFT, the sample data are moved by the set window.

IV.

The signal is divided by finding a case where the difference between the PSD value of the sample data and the original signal is the local minimum region.

V.

Finally, based on the local minimum, compute the start and end points of each cycle.

Through this operation, it is possible to divide the data into a single cycle from the repetitive raw signal, as shown in Figure 4. Hence, the segmented and synchronized data are achieved.

2.3.2. Data Conversion

The input data are fed into the TL model in the form of 2D images to extract autonomous features. As a result, the time domain data must be converted into 2D images. Scalogram images are considered in the proposed method. The upcoming subsections are related to the data conversion into 2D images.

(a)

Scalogram

The scalogram is a time- and frequency-dependent absolute value demonstration of a signal’s continuous wavelet transform (CWT). The scalogram proves more useful than the spectrogram for evaluating real-world signals with features occurring at different scales, such as slow rate variable events interrupted by sudden transients. It can be used to improve time localization for high-frequency, short-term events as well as frequency tracking for low-frequency, longer-duration occurrences. Resampling the signal with a time-scaled and shifted wavelet yields the CWT. Wavelets oscillate and can have a wide range of values. On a prototype wavelet, scalability and transitioning operations are carried out. The CWT scaling shrinks and stretches the prototype wavelet. When the prototype wavelet is shrunk, it generates wavelets with short duration and high frequency, which are useful for detecting transient events. When the prototype wavelet is stretched, long-duration, low-frequency wavelets are produced, which can be used to isolate long-duration, low-frequency events [31]. Figure 5 shows the representation of time domain signals and their scalogram images for simple and welding motions.

(b)

Continuous wavelets transform (CWT)

The wavelet transform outperforms the conventional cosine and Fourier transforms as a time-frequency transform. The continuous wavelet transform (CWT) decomposes complex signal information and extracts interesting patterns by convolution of input sequence with mother wavelet-generated functions. Although the convolution is computed using the short-time Fourier transform, it provides different time-frequency resolutions. [32,33,34]. The wavelet windowing method is used for varying resolution regions. Wavelet decomposition uses a scale rather than a frequency to map a signal into a time-scale plan. This corresponds to the time-frequency plan of the short-time Fourier transform (STFT), with each scale of the time-scale plane representing a different frequency range of the time-frequency plan. The wavelet transforms a signal into sinusoids of varying frequencies, whereas the Fourier transforms a signal into transformed or sized contours from a mother wavelet.

2.4. Transfer Learning Model for Fault Detection

The general framework of TL between the target domain and source domain is shown in Figure 6. Timely fault detection of bearings in the industrial robot is crucial for minimal downtime and uninterrupted operations. The current data obtained from the servomotors were used for bearing fault detection. The current signals are the time series data. The time series data have been converted to scalogram images to better represent the data. These scalogram-based images will be used to develop the FDD model. In the real-world scenario, the availability of a huge amount of labeled data is a big challenge to train, test, and validate the DL model. The shortage of a large amount of data restricts the performance of the deep CNN model. Moreover, training the DL model from scratch is a cumbersome task. The application of transfer learning could mitigate this issue and resolve the issue of training the fault detection model from scratch. The pretrained VGG16 model (already trained on the ImageNet dataset) has been employed for knowledge transfer and fault detection. The VGG6 model has performed efficiently on the ImageNet dataset and comprises multiple convolution blocks, fully connected, and SoftMax layers as the final layer. The weights of the convolutional blocks of the VGG16 model have been transferred to the FDD model. The proposed FDD model is given in the Figure 7. These convolutional blocks (as shown in Figure 7) have been used for extracting features from the input scalogram images developed from the servomotor current signals. These extracted features have been fed to the fully connected layers and SoftMax in the final layer for the bearing fault classification. The weights of the convolutional blocks are frozen, and fully connected layers are kept trainable. The fully connected layers were optimized with the help of the adaptive gradient optimizer. The output layer must classify two states of the bearing faults of the servomotor used in the robotic arm.

3. Results and Discussions

The complete analysis has been carried out on the scalogram images obtained from the current signals of the servomotor of the robotic arm. The dataset has considered various operational modes of industrial robots, including the different motion profiles, operating speeds, and multiple loading conditions. The speed variations from 10% to 100% of the rated speed have been employed during the data collection. Moreover, a variety of loading conditions, including 500 g, 1500 g, 2500 g, and 3500 g, have been considered during the current data acquisition. These 1D current data have been converted into the 2D scalogram, which has been used to develop the FDD model. The complete dataset has been divided into the training, test, and validation dataset. The two states of the servomotor, namely healthy and bearing fault, have been considered in the proposed work. Overall, image-based data for various loading conditions such as 500 g, 1000 g, 1500 g, 2000 g, 2500 g, and 3000 g are considered for training the model based on the simple and welding motion separately. Hence, for both the motions, around 80% of the data is used for the training purpose. To check the generalization capabilities of the model, the highest loading conditions are used for testing and validating the model. For that purpose, 20% of the data is used for testing and validation, considering 10% for testing and 10% for validation. Hence, the health status of the bearing is predicted for the highest loading condition (3500 g) using the trained model with the lower loading condition.

Training in deep architecture is a challenging task, as well as computationally expensive. The weights of the convolutional blocks of the VGG16 have been used for the proposed transfer learning-based fault detection model. The extracted features from the convolutional blocks have been fed to the trainable fully connected layers, with the SoftMax layer as the final layer. The model’s performance was evaluated with the help of a confusion matrix and different performance metrics like accuracy, precision, sensitivity, and F1-score [35].

$$\mathrm{Accuracy}=\frac{TP+TN}{TN+TP+FN+FP}$$

(1)

$$\mathrm{Precision}=\frac{TP}{TP+FN}$$

(2)

$$\mathrm{Sensitivity}=\frac{TP}{TP+FN}$$

(3)

$$F\mathrm{1-score}=\frac{2 TP}{2TP+FP+FN}$$

(4)

where TP, TN, FN and FP represent true positive, true negative, false negative and false positive, respectively.

3.1. Simple Motion

The TL model is used for the FDD of the robotic bearing using simple motion data. It is very complicated to distinguish the healthy and faulty images with the necked eyes as shown in Figure 8. Under the same operating condition of loading and speed, it is hard to see the difference between the healthy and faulty states. However, the pattern for the healthy and faulty states is changing with increasing loading. Hence, the application of the TL model is utilized to predict the health state of the data based on 3500 g from a trained model with lower loading cases.

The TL-based VGG-16 FD model consolidated after approximately 200 epochs. It is demonstrated that the model converged in 200 epochs, as illustrated the training and validation loss curves in Figure 9. For fault classification, an average accuracy of 98% is achieved. Figure 10 depicts the proposed model’s confusion matrix (CM). The CM indicates that healthy and faulty states have been efficiently classified using simple motion data. The performance evaluation parameters such as precision (p), sensitivity (s), and F1-score are shown in

Table 3. Performance metrics for the proposed TL-based model for fault detection using simple motion.

State	Precision ($\mathit{p}$)	Sensitivity ($\mathit{s}$)	$\mathit{F}1$-Score
Faulty	1.0	0.96	0.98
Healthy	0.96	1.0	0.98

. It is noted that the proposed method shows 99.9%, 98%, and 98.08% of training, testing, and validation accuracy, respectively. It is demonstrated that the presented approach of autonomous feature extraction of the TL model showed an effective FDD for the robotic servomotor using simple motion data under variable operating conditions. It is demonstrated that the health state of the higher loading conditions is predicted accurately.

3.2. Welding Motion

Similarly, the simple motion in the case of welding motion under the same operation condition of loading it is complicated to find the fault characteristics for healthy and faulty. However, the fault pattern of the scalogram images is changing with the increased loading on the robotic arm, as shown in Figure 11. In view of that, the electrical motor draws more electrical current with higher loading, hence the pattern changes with higher loading conditions. To provide a generalized FDD model the higher loading data (3500 g) is used for the testing and validation to predict the health state of this data set using the trained model with the lower loading conditions.

The TL-based VGG-16 FD model converged after approximately 200 epochs. Figure 12 depicts the training and validation loss curves. For fault classification, an average accuracy of 99% is achieved. Figure 13 depicts the proposed model’s confusion matrix (CM). The CM shows that conditions such as healthy, and faulty have been significantly classified using welding motion data. The values of performance evaluation parameters such as precision (p), sensitivity (s), and F1-score are shown in

Table 4. Performance metrics for the proposed TL-based FDD for welding motion.

State	Precision ($\mathit{p}$)	Sensitivity ($\mathit{s}$)	$\mathit{F}1$-Score
Faulty	1.0	0.98	0.99
Healthy	0.98	1.0	0.99

.

It is noted that the proposed method shows 99.6%, 99%, and 99.2% of training, testing, and validation accuracy, respectively. It is demonstrated that the proposed approach of autonomous feature extraction of the TL model showed an effective FDD for the robotic servomotor using simple motion data under variable operating conditions. It is demonstrated that the health state of the higher loading conditions is predicted accurately.

To sum up, in the proposed work, the issues related to the DL- and ML-based FDD are solved. In the real-world scenario, the availability of a large amount of labeled data is a big challenge to train, test, and validate the deep model. Hence, the application of TL is applied to overcome the issues related to conventional ML and DL FDD. The above results show the robustness of the proposed approach in terms of simple and welding motion data sets. In addition, the generalization capabilities of the proposed model are validated by using the highest loading conditions for testing and validation purposes. In view of that, the proposed approach can predict the health state of a new data set (unseen data). In future, more cases can be considered by evaluating a comprehensive analysis for predicting the health state of unseen data. Further, the unsupervised model can be proposed for the FDD of the robotic system.

4. Conclusions

In conclusion, the current research work proposed a robust FDD model for the robotic system under the variable working condition of speed, motion, and various loaded. To mimic real environmental conditions, the data were collected from the actual robotic system with real-world operating conditions. For that purpose, a series of different loads were integrated into the robotic manipulator. The electrical current data were collected to overcome the issues of data handling and extra sensor installation. The raw data were preprocessed using data segmentation and data synchronization. The 1D data were converted into 2D images for further processing. The application of TL was used to mitigate the issues related to the DL- and ML-based FDD. The pretrained VGG16 model (already trained on the ImageNet dataset) has been employed for knowledge transfer and fault detection. The VGG6 model performed efficiently on the ImageNet dataset and comprises multiple convolution blocks, fully connected, and SoftMax layers as the final layer. The weights of the convolutional blocks of the VGG16 model were transferred to the fault detection model. Convolutional blocks were used for extracting features from the input scalogram images developed from the servomotor current signals. These extracted features were fed to the fully connected layers and SoftMax in the final layer for the bearing fault classification. The weights of the convolutional blocks are frozen, and fully connected layers are kept trainable. The fully connected layers were optimized with the help of the adaptive gradient optimizer. The output layer classified the health states into healthy and faulty states. The proposed approach shows effective fault detection performance and has the capabilities of generalization under variable working conditions.