Anomaly Detection Methods for Industrial Applications: A Comparative Study

Abstract

Anomaly detection (AD) algorithms can be instrumental in industrial scenarios to enhance the detection of potentially serious problems at a very early stage. Of course, the “Industry 4.0” revolution is fostering the implementation of intelligent data-driven decisions in industry based on increasingly efficient machine learning (ML) algorithms. Most well-known AD methods use a supervised learning approach focusing on fault classification. They assume the availability of labeled data for both normal and anomalous classes. However, in many industrial environments, a labeled set of anomalous data instances is more challenging to obtain than a labeled set of normal data. Hence, this work implements an unsupervised approach based on two different methods using a typical benchmark bearing-fault dataset. The first method relies on the manual extraction of typical vibration metrics provided as input to an ML algorithm. The second one is based on a deep learning (DL) approach, automatically learning latent representation from raw data. The performance metrics demonstrate that both approaches can distinguish the state of a bearing from normal to faulty. DL methodology proves a higher accuracy rate in recognizing faults and a better ability to provide information about the fault size.

1. Introduction

The detection of anomalies is gaining importance in industrial scenarios, driven by the relatively cheap solutions the “Industry 4.0” revolution has made available for data acquisition, storage, cloud computing, and efficient machine learning algorithms. Hence, the growing interest in implementing adequate data-driven decisions is pulling the recent development of several intelligent computing methods for anomaly detection (AD) in industry [1].

Industrial facilities deteriorate over time due to usage and normal wear. Such deterioration must be identified early to prevent possible damages and losses and to maximize machine productivity and remaining useful life (RUL) while reducing maintenance and repair costs. Machinery is then usually instrumented with sensors dedicated to monitoring the performance of industrial components. The data collected by these sensors may include temperature, pressure, vibration, acoustic emission, and motor current [2]. Industrial damage detection typically deals with defects in mechanical components such as motors, turbines, pipeline oil flow, and physical structures.

Most existing machine condition monitoring techniques involve sample/signal analysis and use an indicator to detect fault by comparing it to a predefined alarm threshold. This approach requires expertise in data analysis and knowledge of the machine and its operation. Moreover, the threshold-based technique does not ensure the detection of all possible faults, since some of them involve complex behaviors, such as unwanted oscillations or abnormal increasing/decreasing speed of monitored signals, which remain within thresholds. Recent progress in Artificial Intelligence (AI) technologies has brought several advantages by developing automated fault detection procedures based on available data.

The lack of reliable tagged data may represent a typical issue in many industrial environments, where a labeled set of anomalous data instances covering all possible types of anomalous behavior is usually more challenging to obtain than a labeled set of normal data. In addition, the anomalous behavior is often dynamic (new types of anomalies might arise) and sometimes would result in a catastrophic event, e.g., [3]. For these reasons, semi-supervised and unsupervised learning approaches are the most widely applicable in the industrial domain.

Hence, in this work, an unsupervised approach for AD is investigated and applied to a typical fault detection problem in the industrial field, i.e., the bearings diagnosis. As some of the most relevant rotating machinery components, rolling element bearings play an essential role in adequately functioning various industrial facilities. Bearings are the best location for measuring machinery vibration, where the machine’s primary dynamic loads and forces are applied. Therefore, condition monitoring and fault diagnosis of this critical component can represent the machine’s condition.

The methodology is applied to the Case Western Reserve University (CWRU) benchmark dataset, widely used for developing bearing fault detection approaches. The dataset provides vibration data collected for normal and faulty bearings. Most approaches applied to the CWRU bearing dataset are based on supervised learning and aim to classify different bearing states. This work aims to outline a methodology based on unsupervised learning to monitor online bearing health state and detect possible anomalous behaviors for undertaking preventive countermeasures as soon as the anomaly occurs.

Two different approaches are analyzed for this scope, the first based on the manual extraction of typical vibration metrics to be provided as input to a machine learning (ML) algorithm, the second based on a deep learning (DL) method able to extract useful features from raw data automatically. The two approaches are compared by testing a representative method for each method class, evaluating their effectiveness and efficiency and their strengths and weaknesses in an industrial context.

The article is organized as follows. Section 2 provides a brief background analysis and a literature review of the standard anomaly detection methods applied to the industrial domain. In Section 3, the dataset and the applied methods are described in detail, whereas in Section 4 the obtained results are presented. In Section 5, conclusions are drawn from this study by outlining further research developments.

2. Background and Related Work

As a rule, AD methodologies aim to identify data patterns that differ significantly from expected normal behavior [4]. These nonconforming patterns are most referred to as anomalies or outliers, sometimes interchangeably. Several key aspects need to be considered in developing an adequate AD methodology.

The nature of input data represents a primary factor for an AD technique. Input is generally a collection of data instances. Each data instance can be described using a set of attributes (also referred to as variable, characteristic, feature, or dimension) and might consist of only one attribute (univariate data) or multiple attributes (multivariate data). The attributes can be of different types, such as binary, categorical, or continuous.

Another essential aspect of AD problems regards the type of anomaly. Anomalies can be classified into point, contextual, and collective anomalies [5]. An individual data instance is called a point anomaly if it can be considered anomalous concerning the rest of the data. That is the simplest type of anomaly and the most popular in anomaly detection research. A contextual anomaly occurs if a data instance is anomalous in a specific context but not otherwise. Then, data instances’ contextual and behavioral attributes must be specified as part of the problem formulation. A collective anomaly is defined as a collection of anomalous related data instances for the entire dataset. In this latter case, the individual data instances in the collection may not be anomalies alone, but their occurrence together as a collection is anomalous.

The label of a data instance determines whether that instance can be classified as normal or anomalous. The availability of labeled data for training/validation of models used by AD techniques is usually a significant issue. A human expert often does manual labeling, and obtaining an accurate and well-representative labeled training dataset can be prohibitively expensive. Hence, based on the availability or unavailability of labeled data instances, AD techniques operate in supervised, semi-supervised, and unsupervised modes. Supervised techniques assume the availability of a training dataset that has labeled instances for normal and anomaly classes. A typical approach in such cases is to build a predictive model for normal vs. anomaly classes and use the model to determine to which class any unseen data instance belongs. Unsupervised techniques assume that the training data wholly consist of normal samples, i.e., samples belonging to the class chosen as normal. On the other hand, semi-supervised AD refers to experiments where, in addition to the training set samples belonging only to the normal class, a few labeled anomalies are available and considered by the training objective to help improve the detection [6].

Another key factor for any AD technique is how to report the anomalies. Typically, the outputs may be scores or labels. Scoring methods allow defining a so-called Anomaly Score (AS), a quantitative index assigned to each instance in the test data depending on the observation’s degree of ‘outlierness’. Thus, the output of such methods is a ranked list of anomalies. An analyst may choose to analyze the top few anomalies or use a cutoff threshold to select the most relevant ones. In industrial applications, the AS can be exploited as a health factor for real-time system monitoring. Inspections or other maintenance actions can be undertaken when the AS overcomes a pre-determined threshold. Other methods directly assign a label (normal or anomalous) to each test instance.

AD uses many approaches, including model-based and data-driven methods. However, the complexity of present-day industrial systems and the rising cost of model building lead to limitations on using model-based techniques. Recently, data-driven methods have gained attention because of their ability to derive models without prior knowledge of the application domain, also fostered by recent advances in sensor technologies, data processing, and computation power growth.

Several data-driven algorithms have been applied to AD tasks [7]. Unsupervised rather than supervised methods are often adopted in industrial applications due to the inability, in many realistic cases, to collect a representative set of defective data corresponding to several faulty operations.

Classical ML approaches have been successfully adopted for unsupervised AD. In particular, One-Class Support-Vector Machines (OC-SVMs) are one of the most robust unsupervised outlier detection algorithms and have been successfully used for fault diagnosis by the ML community over the past 15 years. That is due to its excellent generalization ability and high accuracy using a relatively small number of samples [8].

In [9], an automatic method for bearing fault detection and diagnosis is presented based on a one-class SVM trained using only historical data under normal conditions.

The work in [10] proposes novel data-driven architectures trained exclusively on healthy signals, combining LSTM regressors and OC-SVM classifiers, to develop an automated algorithm to identify any abnormal mechanical behavior captured by vibration measurements.

The scientific community has adopted unsupervised DL methods since they outperform conventional ML techniques, which are limited in processing raw data by the need for careful engineering and considerable domain expertise for transforming the data into a suitable internal representation or feature vector. Conversely, DL methods are representation learning methods that allow a machine to be fed with raw data and to automatically discover the representations needed for detection or classification [11]. Several DL approaches in unsupervised AD tasks exploit the autoencoders (AEs) framework. The study in [12] proposes an intelligent fault diagnosis approach that uses a deep neural network (DNN) based on a stacked denoising autoencoder applied to unlabeled data. Authors in [13] propose an unsupervised method for diagnosing faults of electric motors by using a novelty detection approach based on deep AEs. In [14], a comparison of different AE architectures is presented, including vanilla and variational AEs with different types and number of layers, for AD of a real manufacturing industrial furnace.

An innovative DL-based model using AEs is developed in [15], which improves anomaly detection accuracy in bearing condition monitoring.

AD finds extensive use in various application domains, especially industrial damage detection. Among all the publicly available datasets, CWRU Bearing Data Center data have become a standard reference for testing new diagnostic bearing algorithms [16,17]. In most of the research papers that use the CWRU bearing dataset, the aim is to apply unsupervised learning approaches to classify the fault category. Moreover, in the literature, the main concern is the diagnosis of data records. Although classic bearing fault features dominate some signals, others are unclear or display other fault symptoms. The study in [18] provides a benchmark analysis applying three diagnostic methods that use the squared envelope spectrum. The authors found that the data records ranged from easily diagnosable to undiagnosable with applied methods. The ball faults are among the most difficult to diagnose. The same outcome is verified in [19], where envelope analysis is applied to the signal filtered in the frequency band with the most diagnostic information.

In the current research, both OC-SVM and AEs-based approaches are then implemented, assessing and comparing their performance for unsupervised fault detection in the CWRU bearing dataset.

3. Materials and Methods

3.1. Dataset

In order to test the performance of the algorithms used in this study, the CWRU dataset provided by the Case Western Reserve University Bearing Data Center [20] is singled out among all the reference datasets available in the public domain.

The CWRU dataset is a standard benchmark in the context of fault detection. It provides ball-bearing test data for normal and faulty bearings. As shown in Figure 1, the layout of the test rig consists of a 2 hp Reliance Electric motor driving a shaft on which a torque transducer and an encoder are mounted. The test bearings support the motor shaft, and torque is transmitted to the shaft via a dynamometer and an electronic control system.

During the experiments, acceleration was measured in the vertical direction on the housing of the drive-end bearing. Some tests also measured it on the fan-end bearing housing and the motor supporting base plate. Faults deep 0.011 inch and ranging in diameter from 0.007 to 0.028 inch (0.18–0.71 mm) were seeded on the motor’s bearings using electro-discharge machining (EDM). The defects were introduced one at a time on the rolling elements and the inner and outer raceway, and each faulty bearing was reinstalled on the test rig. Vibration data were recorded for motor loads of 0 to 3 hp (motor speeds of 1797 to 1720 rpm) at the sampling frequency of 12 kHz and 48 kHz, depending on the test condition.

For further test setup details, please refer to the CWRU Bearing Data Center website [20].

This study uses drive-end acceleration data collected at the sample rate of 12 kHz in normal baseline conditions and when inner race, outer race, and ball defects are introduced on the fan-end bearing. For the outer race defect, we consider only data related to the ‘orthogonal’ (3.00 o’clock) fault position relative to the load zone. It is worth noticing that, as reported in [18], even though the only load borne by the bearings (in theory) is the static gravitational load from the weight of the shaft and any attached components, there is evidence that there could be a dynamic load superimposed on the static load. The dataset names and details are shown in detail in

Table 1. Dataset names and parameters.

Reference Name	Fault Diameter, in	Motor Load, hp	Approx. Motor Speed, rpm	Data Points
Normal_1	-	1	1772	480,000
Fault_B007_1/Fault_B014_1/Fault_B021_1	0.007/0.014/0.021	1	1772	120,000
Fault_IR007_1/Fault_IR014_1/Fault_IR021_1	0.007/0.014/0.021	1	1772	120,000
Fault_OR007_1/Fault_OR014_1/Fault_OR021_1	0.007/0.014/0.021	1	1772	120,000
Normal_2	-	2	1750	480,000
Fault_B007_2/Fault_B014_2/Fault_B021_2	0.007/0.014/0.021	2	1750	120,000
Fault_IR007_2/Fault_IR014_2/Fault_IR021_2	0.007/0.014/0.021	2	1750	120,000
Fault_OR007_2/Fault_OR014_2/Fault_OR021_2	0.007/0.014/0.021	2	1750	120,000
Normal_3	-	3	1730	480,000
Fault_B007_3/Fault_B014_3/Fault_B021_3	0.007/0.014/0.021	3	1730	120,000
Fault_IR007_3/Fault_IR014_3/Fault_IR021_3	0.007/0.014/0.021	3	1730	120,000
Fault_OR007_3/Fault_OR014_3/Fault_OR021_3	0.007/0.014/0.021	3	1730	120,000

.

Differences in the records can be noted in Figure 2, which shows the raw time signals for the bearing normal condition and each fault category at 3 hp motor load.

3.2. Methodology

The paper presents an exploratory analysis of different methods’ ability to solve a typical anomaly detection problem. For a better understanding, Figure 3 shows the flow chart of the methodology according to the two applied approaches.

The diagram highlights that the techniques operate in unsupervised mode, as the training data entirely consist of samples belonging to the class labeled as normal. In contrast, anomaly classes are used in testing the methods.

The following sections describe the typical data segmentation phase and the two alternative ML and DL approaches.

3.2.1. Data Segmentation

Measurements are first segmented to build windows with a fixed time length. The selection of the window length is an essential issue since it must be set to guarantee, on the one hand, enough data samples for frequency data processing and, on the other hand, a small step size to ensure online monitoring of bearing health state.

The window value is then chosen according to the motor speed and the sampling rate. The motor speed ranges from 1730 to 1772 rpm, meaning approximately 30 rotations/s. As the sampling frequency is set to 12,000 samples/s, about 400 data points are collected in one revolution of the bearing.

For ensuring that the length of a single sample can completely and accurately reflect vibration distribution in the frequency domain with a proper resolution, setting a window length of 800 data points seems appropriate. Given the beneficial effects of data augmentation through overlapping [21], 25% overlap is used, meaning that the stride length includes 600 data points. Data are then segmented into 800 windows for each normal class and 200 windows for each anomaly class.

Figure 4 shows the first overlapping sequences for a typical dataset time series.

3.2.2. Machine Learning Approach

The ML methodology implemented in this study is based on extracting different features from the vibration time series and their use for anomaly detection. Specifically, the features extracted from the raw measurements corresponding to normal operation are used to train an OC-SVM model for AD. The trained model is then tested on the features extracted from new time series data corresponding to normal and anomalous operations.

In more detail, the learning stage allows the nominal vibration behavior model to be constructed. The processing phases can be synthesized as follows:

• A pre-processing converts the time-series acquisitions into a multidimensional scatter plot, each point being a vector of features and representing a time window;
• The dimension of the scatter plot is reduced through Pearson correlation;
• A decision frontier is fitted around the scatter plot by an OC-SVM algorithm.

Once the nominal behavior model is defined, AD is performed on new time series data by applying the same pre-processing and dimension reduction and determining whether the resulting data lie inside or outside the decision frontier.

A custom code is written in Python 3.10.12, using the scikit-learn library (version 1.2.2) [22] to build and train the ML algorithm.

Feature Engineering

Feature engineering uses knowledge about the data to improve the algorithm by applying transformations to the data before providing it as input to the model. The basic idea is making a problem easier by expressing it more simply. A deep understanding of the problem is thus needed [23].

Feature engineering is a general concept that spans different topics, including feature transformation, feature generation, feature selection, feature analysis, and evaluation [24]. Feature transformation refers to constructing new features from existing ones and often uses mathematical mapping. Feature generation, sometimes referred to as feature extraction, is usually about generating new features in addition to the original ones. The final goal is to extract useful information from the raw signals. Unfortunately, not all features are meaningful and informative about machine conditions. So, feature selection is about selecting a small subset of features from a more extensive set. That is very important because providing irrelevant and or highly correlated features as input data may spoil the algorithm’s generalization ability, resulting in overfitting [25]. Furthermore, the reduced dimensionality of the feature set makes the use of specific algorithms computationally feasible and improves their performance. Feature selection may also foster the algorithm interpretability. Feature analysis and evaluation refer to concepts, methods, and measures for evaluating the usefulness of features and feature sets and often are included in the feature selection process. All machine learning tasks, including AD, can use practical feature engineering.

In time series AD, a mainstream approach is extracting multi-domain features through signal processing. The commonly used features in the literature can be extracted from time-domain, frequency-domain, and time-frequency analysis [26,27,28].

Traditional time-domain analysis calculates characteristic features from time series signals as descriptive statistics such as mean, peak, peak-to-peak interval, standard deviation, crest factor, and high-order statistics such as root mean square, skewness, and kurtosis. These features are usually called time-domain features. The calculation formulas of the most common ones used in this study are shown in

Table 2. Time-domain features.

Feature	Formula	Feature	Formula
Max.	${\mathrm{x}}_{\max }=\underset{\mathrm{i}}{\max }\left({\mathrm{x}}_{\mathrm{i}}\right)$	RMS	${\mathrm{x}}_{\mathrm{RMS}}=\sqrt{\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\displaystyle \sum _{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}}{\left({\mathrm{x}}_{\mathrm{i}}\right)}^2}$
Min.	${\mathrm{x}}_{\min }=\underset{\mathrm{i}}{\min }\left({\mathrm{x}}_{\mathrm{i}}\right)$	Crest factor	${\mathrm{x}}_{\mathrm{CF}}=\frac{{\mathrm{x}}_{\mathrm{p}}}{{\mathrm{x}}_{\mathrm{RMS}}}$
Mean	${\mathrm{x}}_{\mathrm{mean}}=\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\displaystyle \sum _{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}}{\mathrm{x}}_{\mathrm{i}}$	Shape factor	${\mathrm{x}}_{\mathrm{SF}}=\frac{{\mathrm{x}}_{\mathrm{RMS}}}{\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\sum }_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}\left|{\mathrm{x}}_{\mathrm{i}}\right|}$
Std.	${\mathrm{x}}_{\mathrm{std}}=\sqrt{\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\displaystyle \sum _{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}}\left({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{mean}}\right)}$	Skewness	${\mathrm{x}}_{\mathrm{skew}}=\frac{\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\sum }_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}{\left({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{mean}}\right)}^3}{{\left[\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\sum }_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}{\left({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{mean}}\right)}^2\right]}^{3/2}}$
Peak	${\mathrm{x}}_{\mathrm{p}}=\underset{\mathrm{i}}{\max }\left(\left|{\mathrm{x}}_{\mathrm{i}}\right|\right)$	Kurtosis	${\mathrm{x}}_{\mathrm{kurt}}=\frac{\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\sum }_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}{\left({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{mean}}\right)}^4}{{\left[\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\sum }_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}{\left({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{mean}}\right)}^2\right]}^2}$
Peak-to-peak	${\mathrm{x}}_{\mathrm{p}2\mathrm{p}}={\mathrm{x}}_{\max }-{\mathrm{x}}_{\min }$	Impulse factor	${\mathrm{x}}_{\mathrm{IF}}=\frac{{\mathrm{x}}_{\mathrm{p}}}{\frac{1}{{\mathrm{N}}_{\mathrm{s}}}{\sum }_{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{s}}}\left|{\mathrm{x}}_{\mathrm{i}}\right|}$

.

Frequency analysis is the most used method for analyzing a vibration signal utilizing its frequency spectrum [29]. The most basic type of frequency analysis is an FFT, or Fast Fourier Transform, which converts a signal from the time domain into the frequency domain. In a complex signal, the FFT helps engineers determine the frequencies being excited and their amplitudes, highlighting the changes in frequency and amplitude and the harmonic excitation in a defined frequency range. In order to view the energy distribution across the frequency spectrum, the PSD, or Power Spectral Density, must be calculated from the FFT. The PSD is the measure of signal’s power content over frequency and the most common tool for analyzing broadband random vibration. PSD and FFT are tools for measuring and analyzing a signal’s frequency content. However, whereas the FFT transfers time data to the frequency domain, highlighting changes in frequency values, the PSD takes another step and calculates the power, or strength, of the frequency content. The magnitude of the PSD is normalized to the frequency bin width. Therefore, the resulting value is independent of the bandwidth and suitable for analyzing and comparing random vibration signals, whose frequency spectra can be overlaid and compared independently of the spectral resolution. Among all the commonly used frequency-domain features, in this study, the following features, calculated from the PSD frequency spectrum, are considered:

• The peak frequency;
• The peak amplitude;
• The RMS level in the low-frequency range (below 65 Hz);
• The RMS level in the mid-frequency range (between 65 Hz and 300 Hz);
• The RMS level in the high-frequency range (above 300 Hz);
• The overall RMS level.

The calculation formulas of the frequency domain features are reported in

Table 3. Frequency-domain features.

Feature	Formula
Peak frequency	${\mathrm{f}}_{\mathrm{peak}}={\mathrm{f}}_{\mathrm{k}} : \underset{\mathrm{k}}{\max }\left({\mathrm{s}}_{\mathrm{k}}\right)$
Peak amplitude	${\mathrm{s}}_{\mathrm{peak}}=\underset{\mathrm{k}}{\max }\left({\mathrm{s}}_{\mathrm{k}}\right)$
${\mathrm{RMS}}_{\mathrm{f} < 65 \mathrm{Hz}}$	${\mathrm{s}}_{(\mathrm{f} < 65 \mathrm{Hz})}=\sqrt{\frac{1}{{\mathrm{k}}_{65}}{\displaystyle \sum _{\mathrm{k}=1}^{{\mathrm{k}}_{65}}}{\left({\mathrm{s}}_{\mathrm{k}}\right)}^2}$
${\mathrm{RMS}}_{65 \mathrm{Hz} < \mathrm{f} < 300 \mathrm{Hz}}$	${\mathrm{s}}_{(65 \mathrm{Hz} < \mathrm{f} < 300 \mathrm{Hz})}=\sqrt{\frac{1}{{{\mathrm{k}}_{300}- \mathrm{k}}_{65}}{\displaystyle \sum _{\mathrm{k}={\mathrm{k}}_{65}}^{{\mathrm{k}}_{300}}}{\left({\mathrm{s}}_{\mathrm{k}}\right)}^2}$
${\mathrm{RMS}}_{\mathrm{f} > 300 \mathrm{Hz}}$	${\mathrm{s}}_{(\mathrm{f} > 300 \mathrm{Hz})}=\sqrt{\frac{1}{\mathrm{K}-{\mathrm{k}}_{300}}{\displaystyle \sum _{\mathrm{k}={\mathrm{k}}_{300}}^{\mathrm{K}}}{\left({\mathrm{s}}_{\mathrm{k}}\right)}^2}$
${\mathrm{RMS}}_{\mathrm{overall}}$	${\mathrm{s}}_{\mathrm{f}}=\sqrt{\frac{1}{\mathrm{K}}{\displaystyle \sum _{\mathrm{k}=1}^{\mathrm{K}}}{\left({\mathrm{s}}_{\mathrm{k}}\right)}^2}$

, where s(k) is the PSD spectrum for k = 1, 2, …, K, K is the number of spectrum lines, and f_k is the frequency value of the kth spectrum line.

Time-frequency analysis allows obtaining features usually extracted by wavelet transform (WT) or empirical mode decomposition (EMD), which can reflect the health status of machines under non-stationary operating conditions.

Since the CWRU bearing-test data do not include any transient condition, only time-domain and frequency-domain features are extracted and fed as input into the ML algorithm. Before that, the most discriminative features must be selected from the feature set, and redundant features must be discarded to improve the model performance and avoid the curse of dimensionality. Here, the Pearson correlation matrix is computed. It measures the strength and direction of the relationship between two variables and thus allows the identification of the correlated features. The screening process analyzes the correlation coefficients of all the couples of features. For each couple, one of the features is discarded if they are highly correlated, and the model performance is not penalized. The least redundant features are chosen, reducing the total number of features from 18 to 10. The ten chosen features are RMS, max, peak-to-peak, skewness, kurtosis, and shape factor as time-domain features, peak amplitude, RMS level in the mid-frequency range, RMS level in the high-frequency range, and overall RMS level as frequency domain features.

OC-SVM Algorithm

Support Vector Machine (SVM) is one of the most successful machine learning techniques that can be used in various classification applications. The technique classifies data by finding the hypersurface that best separates the features into different domains. If the training data are not linear, it uses a kernel function to map it into a higher-dimension space. The instances closest to the separating hyperplane are called support vectors. In contrast to traditional SVM, OC-SVM is an unsupervised method that does not require target labels for the training process, as it aims to determine the behavior of data from a single known class, the target class, and tag the elements that lie on the other side of the decision boundary as outliers. For more detailed information, refer to Schölkopf et al. [30,31].

This study implements an OC-SVM classifier on the selected feature set for each motor load condition. A radial basis function (RBF) is used as the kernel function because of its ability to model complex and non-linear decision boundaries [32]. The RBF kernel measures the similarity between two data points as a function of the Euclidean distance between them. The kernel function is defined as:

K(x, y) = exp(−γ ||x − y||²),

(1)

where x and y are two data points, γ is the kernel coefficient that determines the width of the kernel function, and ||x − y|| is the Euclidean distance between x and y. Choosing the correct value of γ is crucial for achieving good performance with the RBF kernel. γ defines the range of influence of a single training point. It can be seen as the inverse of the radius of influence of samples selected by the model as support vectors. A more considerable γ value makes the kernel function more peaked, leading to a more complex decision boundary that better captures the details of the training data; conversely, a more minor γ value results in a smoother decision boundary that may be more generalizable to new data. In addition to γ, the RBF kernel has another hyperparameter called ν. This regularization parameter controls the trade-off between achieving a good fit to the training data and a simple decision boundary. Larger values of ν allow for more complex decision boundaries, which can lead to overfitting, whereas smaller values of ν may result in underfitting. Therefore, both parameters control complexity in some way.

In this case, the default value of γ in scikit-learn is selected as the kernel coefficient (equal to the inverse of the product of the number of features and the variance of the data) since the dataset is not scaled to have a unit standard deviation. Because only the normal class needs to be modeled, the value of ν is tuned to the approximate ratio of outliers in the data. In particular, coherently with [30], ν = 0.01 is chosen to be consistent with the estimated fraction of abnormal instances. The AS, by definition, is computed through the scoring function of the samples.

In order to differentiate the classifier depending on the specific anomaly class to be detected, several OC-SVM models are trained in parallel by changing the input features while keeping the model tuning parameters fixed. That led to identifying, for each fault category, the set of features shown in

Table 4. Reduced set of features for each anomaly class.

Anomaly Class	Features
Ball Fault	xRMS, xp2p, xskew, s(65Hz<f<300Hz), s(f>300Hz)
Inner Race Fault	xmax, xRMS, xp2p, xskew, xSF, speak, s(f>300 Hz)
Outer Race Fault	xmax, xRMS, xp2p, xkurt, s(f>300 Hz), sf

, ensuring the best model performance.

The feature set is split into training and testing sets, as reported in

Table 5. Data splitting for OC-SVM algorithm.

Set	Normal Samples	Anomalous Samples
Training	600	0
Test	200	200

. The algorithm is trained using only normal data samples and tested using both normal and anomalous data samples. Input consists of multivariate data, i.e., the multiple features calculated from the acceleration time series at the drive end of the motor housing.

3.2.3. Deep Learning Approach

DL methods have become increasingly popular in recent years because of their success in many domains, such as image classification, speech recognition, and natural language processing. The great success of DL mainly comes from specially designed structures of deep nets, which can learn discriminative non-linear features. Hence, most existing DL methods can be used as powerful feature learning/extraction tools, as the latent features extracted by DL algorithms are newly learned representations.

DL approach is also used in AD problems to detect time series anomalies. Among the different methods for time series data in the literature, the reconstruction methods [33] build a model of normal behavior by encoding subsequences of a normal training time series in a (low dimensional) latent space. In order to detect anomalies in a test time series, subsequences of the test series are reconstructed from the latent space, and their values are then compared to the original series values. Since the model is trained only on normal data, the model cannot reconstruct anomalous subsequences in the test series. Therefore, the difference between the original and the reconstructed subsequences can provide an anomaly score.

A representative method of this category used for the present evaluation is the autoencoder (AE) model. An AE is a neural network trained to learn latent representation able to reconstruct its input [34]. Generally, an AE comprises two parts, i.e., an encoder f(·) and a decoder g(·). An illustration of AE is shown in Figure 5.

The encoder maps the input x $\in$ R^m to latent representation h $\in$ R^d as h = f (x) and f (·) is usually a one-layer neural network, i.e., f (x) = s (Wx + b), where W $\in$ R^{d x m} and b $\in$ R^d are the weights and bias of the encoder. s (·) is a non-linear function such as sigmoid and tanh. A decoder maps back the latent representation h into a reconstruction $\stackrel{\mathrm{~}}{\mathrm{x}}$ $\in$ R^m as $\stackrel{\mathrm{~}}{\mathrm{x}}$ = g (h) and g (·) is given as g (h) = s (W′h + b′), where W′ $\in$ R^{m x d} and b′ $\in$ R^m are the weights and bias of the decoder. Note that the prime symbol does not indicate matrix transposition. The parameters of the autoencoder θ = {W, b, W′, b′} are optimized to minimize the reconstruction error. Depending on the distribution assumptions of the input, the reconstruction error can be measured in different ways. The most widely used reconstruction error is the squared error $\mathcal{L}\left(\mathrm{x},\stackrel{\mathrm{~}}{ \mathrm{x}}\right)={‖\mathrm{x} -\stackrel{\mathrm{~}}{\mathrm{x}}‖}_2^2$.

The autoencoder can be used to extract valuable features by forcing the latent representation h to have a smaller dimension than x, i.e., d < m. Learning such a representation forces the autoencoder to capture the most salient features of the training data.

An AE algorithm is trained over the normal vibration data for each considered motor load condition to build a model of bearing nominal behavior. The algorithm is coded in Python 3.10.12, using the Keras computational library (version 2.12.0) [35] to implement the AE architecture.

After the segmentation process described in Section 3.2.1, the dataset is split into training and testing sets. The model is trained using only normal data samples and tested over normal and anomalous data samples.

Table 6. Data splitting for AE algorithm.

Set	Normal Samples	Anomalous Samples	Size
Training	600	0	800
Test	200	200	800

reports the data splitting information. Now, the input consists of univariate data, directly the vibration time series collected by the accelerometer at the drive end of the motor housing.

In the AE architecture, input and output vectors have dimension T = 800, whereas the latent space has dimension D = 24. The encoder and the decoder are constituted by six fully connected layers, consisting of a complete feed-forward connection without activation function, as detailed in

Table 7. The fully connected AE structure.

Component	Architecture
Encoder x800 → h24	→ FC800 → FC400 → FC200 → FC100 → FC50 → FC24 →
Decoder h24 → $\stackrel{\mathrm{~}}{\mathrm{x}}$800	→ FC24 → FC50 → FC100 → FC200 → FC400 → FC800 →

.

After training, a limit threshold to the reconstruction error must be adequately set to evaluate the algorithm’s performance.

One of the simplest threshold selection methods uses the three-sigma rule of thumb [36], expressing a conventional heuristic. Almost all values will lie within three standard deviations of the mean for normally distributed data. Thus, the 99.7% probability can be treated as near certainty. Therefore, data points that fall outside three-sigma limits can be considered outliers. Figure 6 shows the threshold definition in this work, set to three standard deviations of the mean of the loss computed over training normal samples.

4. Results and Discussion

First, the models’ performances are assessed using some of the most used evaluation metrics listed in the following.

• Accuracy: the fraction of correctly predicted instances over the total number of instances.

  $$\mathrm{Accuracy}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}+\mathrm{TN}+\mathrm{FN}},$$

  (2)
• Precision: the fraction of positive instances the model correctly predicts, useful when the cost of false positives is high.

  $$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}},$$

  (3)
• Recall: the fraction of positive instances correctly predicted by the model out of all the positive instances in the data, useful when the cost of false negatives is high.

  $$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},$$

  (4)
• F1 Score: the harmonic mean of precision and recall, a balanced measure considering false positives and false negatives.

  $$\mathrm{F}1 \mathrm{Score}=\frac{2 \mathrm{\ast } \mathrm{Precision} \mathrm{\ast } \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}},$$

  (5)
• AUC-ROC: the “Area Under the Receiver Operating Characteristic” curve, i.e., the probability that a randomly selected positive instance will be ranked higher than a randomly selected negative instance [37].

Table 8. OC-SVM and AE performance metrics in detecting inner race defects.

Inner Race Faults (IR007/IR014/IR021)
Method	Motor Load, hp	Performance Metrics
		Accuracy	Precision	Recall	F1 Score	AUC
OC-SVM	1	0.993/0.996/0.996	0.991/0.991/0.991	0.995/1/1	0.993/0.995/0.995	0.993/0.996/0.996
		±0.004/±0.004/±0.004	±0.008/±0.008/±0.008	0/0/0	±0.004/±0.004/±0.004	±0.004/±0.004/±0.004
	2	0.995/0.997/0.997	0.994/0.994/0.994	0.995/1/1	0.994/0.997/0.997	0.995/0.997/0.997
		±0.007/±0.007/±0.007	±0.013/±0.013/±0.013	0/0/0	±0.007/±0.007/±0.007	±0.007/±0.007/±0.007
	3	0.996/0.996/0.996	0.991/0.991/0.991	1/1/1	0.995/0.995/0.995	0.996/0.996/0.996
		±0.003/±0.003/±0.003	±0.007/±0.007/±0.007	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
AE	1	0.984/0.984/0.984	0.969/0.969/0.969	1/1/1	0.984/0.984/0.984	0.984/0.984/0.984
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
	2	0.987/0.987/0.987	0.975/0.975/0.975	1/1/1	0.987/0.987/0.987	0.987/0.987/0.987
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
	3	0.981/0.981/0.981	0.962/0.962/0.962	1/1/1	0.981/0.981/0.981	0.981/0.981/0.981
		±0.010/±0.010/±0.010	±0.018/±0.018/±0.018	0/0/0	±0.009/±0.009/±0.009	±0.009/±0.009/±0.009

,

Table 9. OC-SVM and AE performance metrics in detecting outer race defects.

Outer Race Faults (OR007/OR014/OR021)
Method	Motor Load, hp	Performance Metrics
		Accuracy	Precision	Recall	F1 Score	AUC
OC-SVM	1	0.990/0.990/0.990	0.979/0.979/0.979	1/1/1	0.989/0.989/0.989	0.990/0.990/0.990
		±0.007/±0.007/±0.007	±0.014/±0.014/±0.014	0/0/0	±0.007/±0.007/±0.007	±0.007/±0.007/±0.007
	2	0.991/0.991/0.985	0.982/0.982/0.982	1/1/0.987	0.991/0.991/0.984	0.991/0.991/0.985
		±0.007/0.007/0.004	±0.014/±0.014/±0.014	0/0/±0.006	±0.007/±0.007/±0.005	±0.007/±0.007/±0.004
	3	0.990/0.990/0.990	0.980/0.980/0.980	1/1/1	0.990/0.990/0.990	0.990/0.990/0.990
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
AE	1	0.984/0.984/0.984	0.969/0.969/0.969	1/1/1	0.984/0.984/0.984	0.984/0.984/0.984
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
	2	0.987/0.987/0.987	0.975/0.975/0.975	1/1/1	0.987/0.987/0.987	0.987/0.987/0.987
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
	3	0.981/0.981/0.981	0.962/0.962/0.962	1/1/1	0.981/0.981/0.981	0.981/0.981/0.981
		±0.010/±0.010/±0.010	±0.018/±0.018/±0.018	0/0/0	±0.009/±0.009/±0.009	±0.009/±0.009/±0.009

and

Table 10. OC-SVM and AE performance metrics in detecting ball defects.

Ball Faults (B007/B014/B021)
Method	Motor Load, hp	Performance Metrics
		Accuracy	Precision	Recall	F1 Score	AUC
OC-SVM	1	0.856/0.981/0.993	0.986/0.986/0.986	0.783/0.975/1	0.873/0.981/0.993	0.856/0.981/0.993
		±0.024/±0.005/±0.004	±0.007/±0.007/±0.007	±0.029/±0.006/0	±0.019/±0.005/±0.004	±0.024/±0.005/±0.004
	2	0.665/0.993/0.979	0.985/0.985/0.985	0.603/1/0.974	0.747/0.992/0.979	0.665/0.993/0.979
		±0.056/±0.006/±0.004	±0.012/±0.012/±0.012	±0.046/0/±0.015	±0.032/±0.006/±0.004	±0.056/±0.006/±0.004
	3	0.855/0.993/0.993	0.986/0.986/0.986	0.781/1/1	0.872/0.993/0.993	0.855/0.993/0.993
		±0.013/±0.005/±0.005	±0.010/±0.010/±0.010	±0.016/0/0	±0.011/±0.005/±0.005	±0.013/±0.005/±0.005
AE	1	0.984/0.984/0.984	0.969/0.969/0.969	1/1/1	0.984/0.984/0.984	0.984/0.984/0.984
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
	2	0.987/0.987/0.987	0.975/0.975/0.975	1/1/1	0.987/0.987/0.987	0.987/0.987/0.987
		±0.003/±0.003/±0.003	±0.006/±0.006/±0.006	0/0/0	±0.003/±0.003/±0.003	±0.003/±0.003/±0.003
	3	0.981/0.981/0.981	0.962/0.962/0.962	1/1/1	0.981/0.981/0.981	0.981/0.981/0.981
		±0.010/±0.010/±0.010	±0.018/±0.018/±0.018	0/0/0	±0.009/±0.009/±0.009	±0.009/±0.009/±0.009

show the values of the evaluation metrics across the two different approaches for each anomaly class and load condition. Test performances are computed on five different random train-test splits; therefore, metrics values are here expressed in terms of mean and standard deviation. The results reveal that, on average, both methods perform well. In almost all cases, OC-SVM exhibits the best performance in detecting bearing inner race and outer race defects.

At the same time, the AE reconstruction method performs better for the diagnosis of ball fault cases. For this type of anomaly, the ML approach shows lower accuracy and recall values when the fault diameter is 0.007 inch. It is worth noticing that, unlike OC-SVM, the approach based on AEs allows 100% recall (FN = 0) for all experiments, presenting a 100% rate in recognizing faults.

For better comprehension, the simulation results of compared methods are also analyzed in terms of AS and confusion matrices, which help to understand the types of errors the model makes, showing the number of true positives, true negatives, false positives, and false negatives. Specifically, the OC-SVM score samples and the AE loss are used as anomaly scores, and their values are plotted for 200 normal and 200 faulty test samples. Some of the most representative results are shown in Figure 7, Figure 8 and Figure 9, where each black dot corresponds to a normal sample. In contrast, a blue, green, and red one corresponds to an abnormality, and yellow diamond points, if present, mark false positives and false negatives.

Confusion matrices in Figure 10, Figure 11 and Figure 12 better detail the results in Table 8, Table 9 and Table 10. As expected, the worst case can be related to the OC-SVM method presenting many false negatives (FN = 51) when detecting the bearing ball fault of 0.007 inch -size. The resulting low recall value means the algorithm confuses the small-size ball defects with the normal mode.

From a detailed data analysis, both approaches are considered accurate in distinguishing high-severity faults from normal behavior. In Figure 8, the red dot corresponding to the highest AE loss (sample # 261—Fault_IR021_2) is the red dot with the most considerable distance to the OC-SVM score of the normal observations. The analysis in the time and frequency domain of sample # 261 in Figure 13 highlights high-amplitude vibration, mainly in the range of 3–4 kHz, for the 0.021-inch size fault class, much higher than vibration amplitudes shown by the same sample number corresponding to the anomaly classes with a lower fault size (Fault_IR014_2 and Fault_IR007_2). This evidence is also reflected by the values of the features given as input to the ML model and shown in Figure 14.

On the other hand, the ML algorithm’s difficulty in correctly ranking the fault severity is highlighted. Figure 15 demonstrates differences in the 3–4 kHz amplitudes of the two compared signals, not detected by OC-SVM that assigns a similar AS to both samples. The AE reconstruction method, instead, is more effective in detecting even slight differences in the vibration signature. The scarce sensitivity of the OC-SVM method is probably due to the small set of features extracted, which can only capture some helpful information from the raw signals.

The unsupervised recognition of ball faults finally demonstrates the best overall performances of AEs-based methods. As reported in the literature [18,19], the ball faults are certainly the most difficult to diagnose, as in many cases, they do not give the classic spectrum symptoms when using established bearing diagnostic techniques. That explains the poor performance of OC-SVM in detecting small-size ball defects since it uses traditional time-domain and frequency-domain characteristic features.

So, the ML algorithm underperforms in comparison with the AE reconstruction method. Of course, the latter needs to be further improved. Varying model architecture regarding types of layers and hyperparameter settings during the training could reduce the false alarm rate of normal samples to zero.

Despite the lower performance, the strength of OC-SVM is the capability to provide additional information, giving insights into the possible root causes of the faults. As shown in Table 4, the SVM models are built on a reduced set of features, which differs for each analyzed anomaly class. That can help the anomaly diagnosis by identifying the features most sensitive to a specific type of fault. The real-time monitoring of their value ensures the detection and diagnosis of the fault, providing information about the defect’s location. Evidence is given in Figure 16, where the inner race fault signature appears better described by peak amplitude. At the same time, kurtosis and RMS levels in the mid-frequency range characterize outer race and ball fault vibration, respectively.

5. Conclusions

The proposed research study aims to explore using two different unsupervised approaches for AD on a typical benchmark industrial dataset, i.e., the CWRU bearing fault dataset. The first approach is based on the manual extraction of typical vibration metrics to be provided as input to an ML model based on the OC-SVM algorithm. In contrast, the second DL approach automatically uses AEs to learn latent representation from raw data.

The evaluation metrics demonstrate that OC-SVM exhibits, in almost all cases, better performances than AEs in detecting bearing inner race and outer race defects, but it shows lower values of accuracy and recall for ball fault detection. Conversely, the approach based on AEs allows 100% recall for all experiments, presenting a 100% detection rate for all anomaly types, including the not easily diagnosable ball fault. Both approaches accurately distinguish high-severity faults from normal behavior, even though the ML algorithm presents a lower ability to rank fault severity properly.

So, the AE reconstruction method outperforms the ML algorithm. The main drawback of the DL method is the lack of interpretability of its internal operations, operating as a black box machine. On the other hand, depending on the anomaly class, selecting different feature sets allows identifying the most relevant features to detect a specific type of fault, improving the diagnosis results. That would make the process explainable to maintenance operators.

Future work may focus on improving the DL model to eliminate the false positive rate and studying methodologies to make the AE interpretable. The OC-SVM performances should also be improved to extend the number of extracted features, enabling the capture of all relevant and helpful information from the raw signals. Another development should make the feature selection process unrelated to the test set performance.