Noise-to-Norm Reconstruction for Industrial Anomaly Detection and Localization

Abstract

Anomaly detection has a wide range of applications and is especially important in industrial quality inspection. Currently, many top-performing anomaly detection models rely on feature embedding-based methods. However, these methods do not perform well on datasets with large variations in object locations. Reconstruction-based methods use reconstruction errors to detect anomalies without considering positional differences between samples. In this study, a reconstruction-based method using the noise-to-norm paradigm is proposed, which avoids the invariant reconstruction of anomalous regions. Our reconstruction network is based on M-net and incorporates multiscale fusion and residual attention modules to enable end-to-end anomaly detection and localization. Experiments demonstrate that the method is effective in reconstructing anomalous regions into normal patterns and achieving accurate anomaly detection and localization. On the MPDD and VisA datasets, our proposed method achieved more competitive results than the latest methods, and it set a new state-of-the-art standard on the MPDD dataset.

1. Introduction

In the field of computer vision, visual anomaly detection is a crucial issue that involves the process of detecting and pinpointing data instances that deviate from the expected or standard observations. In the past years, Anomaly detection had a wide range of applications in fields such as industrial quality inspection [1,2,3], medical diagnosis [4], and video surveillance [5,6]. In industrial production processes, product quality is highly susceptible to various factors, such as existing technologies and working conditions. Surface defects are the most visible manifestation of the impact on product quality. Therefore, surface defect detection is crucial to ensure product qualification and reliability. Anomaly detection plays a vital role in industrial quality inspection by identifying and locating defects in product appearance, thereby improving product quality and ensuring compliance with standards.

In recent years, with the rapid development of the computer vision field, deep learning-based methods have become an effective solution for industrial quality inspection. However, due to the limited number of anomaly samples and the difficulty of including all types of anomalies in a limited number of samples, and the labor-intensive data labeling process, supervised anomaly detection methods are severely limited. As a result, most recent research has focused on unsupervised learning methods, which aim to achieve anomaly detection without requiring prior information on anomaly samples. Unsupervised learning-based anomaly detection has become a promising research trend for the future.

In unsupervised anomaly detection methods, there are mainly two types of approaches: feature embedding-based and reconstruction-based methods. Feature embedding-based methods [7,8,9,10,11] that utilize pre-trained models to extract image features and realize the measurement or comparison of features by feature modeling, have been widely used in the field of unsupervised anomaly detection. However, feature embedding-based methods rely on pre-trained models on additional datasets such as ImageNet, and their performance may significantly decrease when the features of industrial images differ significantly from those of the pre-training dataset. Additionally, feature embedding-based methods are highly dependent on the consistency of the placement of the target and the shooting angle. Otherwise, false detections are likely to occur. In contrast, reconstruction-based methods [12,13,14,15] do not have this limitation and do not require additional training data, making them suitable for various scenarios. Reconstruction-based methods exhibit better anomaly detection and localization performances for randomly placed objects. However, reconstruction-based methods also have limitations. They may not be able to achieve complete reconstruction when dealing with images with complex structures and textures. In addition, reconstruction-based methods may result in invariant reconstruction of anomalous regions due to their strong reconstruction ability.

To address the aforementioned issues, we propose a reconstruction-based method using the noise-to-norm paradigm, which trains the reconstruction network on noisy images as input, significantly improving the network’s reconstruction ability. Additionally, the introduction of noise disturbs the anomalous regions, making it difficult to distinguish them from normal patterns, thus solving the problem of invariant reconstruction of anomalous regions.

Our proposed reconstruction model is based on M-net [16] and employs a multiscale fusion structure. Before being fed into the reconstruction network, the noisy image is down-sampled to varied sizes to enlarge the model’s receptive field, providing better robustness to anomalous regions of diverse sizes. The reconstruction network comprises three parts: an encoder, a decoder, and a feature fusion module; both the encoder and decoder contain residual attention modules and skip connections between them. The feature fusion module fuses the multiscale features to generate the reconstructed image.

Numerous experiments on the MPDD [2] and VisA [3] datasets have demonstrated that the proposed end-to-end anomaly detection method has excellent performance.

The main contributions of this study are summarized as follows:

• We introduce a novel unsupervised anomaly detection method based on the noise-to-norm paradigm.
• We propose a residual attention module that can be embedded in the encoder and decoder to achieve high-quality reconstruction of noisy images.
• Our method achieves state-of-the-art (SOTA) performance on the MPDD dataset.

2. Related Work

Unsupervised learning addresses the high annotation costs and difficulty in collecting negative samples, making it the mainstream method for image anomaly detection. Unsupervised learning methods can be divided into two main categories: reconstruction-based and feature embedding-based methods.

2.1. Feature Embedding-Based Methods

Feature embedding-based methods aim to determine a feature distribution that can distinguish between normal and anomalous samples. Typically, these methods use a pre-trained network as a feature extractor to extract shallow features from images. By fitting normal sample features to a Gaussian distribution, Mahalanobis distance is a common method to calculate the anomaly scores [7,17] between the test set samples and the Gaussian distribution, to estimate the anomaly localization. Research in [8] employed a coreset-subsampled memory bank to ensure low inference cost at higher performance. Some studies attach a normalizing flow module to the feature extractor [10,11,18], features were first extracted and a normalizing flow module was utilized to enable transformations between data distributions and well-defined densities. Subsequently, anomaly detection and localization were performed based on the probability density of the feature map.

In general, feature embedding-based methods have achieved better results on the MVTec AD [1] dataset than those of the reconstruction-based methods because of their powerful representation capability of deep features. However, they rely on the uniformity of an object’s location, which makes optimization difficult for cases in which the object’s position varies significantly.

2.2. Reconstruction-Based Methods

The reconstruction-based method trains an encoder and a decoder to reconstruct images with low dependence on pre-trained models. This method aims to train a reconstruction model that works well on positive samples but poorly on anomalous regions and achieves anomaly detection and localization by comparing the original image with the reconstructed image. Early studies used Autoencoders [13,19,20] for image reconstruction, whereas some methods employed a generative adversarial network [12,14,21] to obtain better reconstruction performance. However, there is a problem of overgeneralization, which can lead to an accurate reconstruction of anomalous regions. To address this issue, some researchers proposed a method based on image inpainting [15,22,23,24], in which masks are used to remove parts of the original image, preventing the reconstruction of anomalous regions. However, for images with complex structures and irregular textures, excessive loss of the original information may limit the reconstruction ability and cause many false positives in normal regions.

3. Method

3.1. Overview

The proposed anomaly detection framework is based on the noise-to-norm paradigm, as shown in Figure 1.

Specifically, we introduce random Gaussian noise to corrupt the original image, and the process of adding noise $ϵ$ is defined as follows:

$$x=(1-\lambda )x_0+\lambda ϵ,ϵ\sim N(0.5,0.5)$$

(1)

where $\lambda \in (0,1),x_0$ is the data obtained by normalizing each channel of the original image according to a Gaussian distribution $(\mu =0.5,\sigma =0.5)$. We add random noise generated from the same Gaussian distribution to the original image using weighted blending, thereby allowing us to control the degree to which the noise corrupts the original image. In contrast to the methods that simulate anomalies [20], our approach of adding noise is not intended to simulate anomalies. Instead, its purpose is to completely obscure the distinguishable appearance of anomalous regions, allowing the reconstruction network to transform the anomalous image into a normal image.

After adding noise, the images were down-sampled to varied sizes to serve as multiple inputs. These inputs were then utilized by the reconstruction network to generate anomaly free images. During the training phase, only anomaly free samples were used to train the reconstruction network. The reconstructed images were compared to the original images using a loss function, and the reconstruction capability of the model was continuously improved. During the inference phase, anomaly localization was achieved by generating an anomaly map that captured pixel-level differences between the reconstructed and original images. The specific details of the reconstructed network are described below.

3.2. Reconstruction Network

As we know, U-Net [25] and its variants have been widely used in deep learning models for image segmentation and other fields [26,27,28,29]. Our proposed network is based on the M-net [16], which originated from the field of image segmentation and has been proven to be effective in the domain of denoising. The overall architecture of the proposed reconstruction network is shown in Figure 2. Inspired by the SRMnet [30], we incorporated pixel shuffle operations into the encoder and decoder for upsampling and downsampling; this allows us to effectively manage resolution changes in the network and improve the reconstruction quality. The residual attention modules were merged after concatenating the features to enhance the feature representation and capture the relevant information. The encoder and decoder were connected through skip connections to facilitate the flow of information between different feature levels. The multiscale features were combined in the feature fusion module to generate the final reconstructed image. This design enables the network to effectively capture anomalies and produce high-quality reconstructions.

3.2.1. Residual Attention Module

The Residual Attention Modules are integrated after the concatenation of features to enhance feature representation and capture relevant information. These modules leverage residual connections and attention mechanisms to selectively emphasize notable features and suppress irrelevant ones. By focusing on informative regions and enhancing feature discrimination, the residual attention modules improve the network’s ability to generate high-quality reconstructions. In addition, the residual connections address the issue of vanishing gradients. By propagating gradients more effectively through the network, the residual connections enable faster convergence and improve the accuracy of the model. The specific structure of the residual attention module is shown in Figure 3. It comprises global pooling, convolutional, and activation layers. In both pathways, a $1\times 1$ convolutional layer is employed to adjust the number of feature channels.

3.2.2. Selective Kernel Feature Fusion (SKFF)

Our decoder generates four feature maps with different resolutions, and we employed the SKFF [31] module for feature fusion. The SKFF allows for the selection of different convolutional kernels at different spatial positions to facilitate the fusion of features from different scales, enabling the integration of multiscale reconstruction features. This approach avoids directly connecting each feature map and instead aggregates weighted features, addressing the issues of a large number of parameters and higher computational complexity in the M-net.

3.3. Metric Function

We employed a metric function that combines the MS-SSIM and ${\ell }_1$ proposed by Hang Zhao et al. in [32]. SSIM [33] is a widely used indicator for measuring the structural similarity between images. SSIM mainly considers three key features of images: luminance, contrast, and structure. MS-SSIM uses multiple different Gaussian filters ($\sigma ={\sigma }_1,\dots ,{\sigma }_M$) on the basis of SSIM. Specifically, for input images x and y of the same size, the luminance similarity of each pixel p is defined as:

$$l\left(p\right)=\frac{2{\mu }_x{\mu }_y+C_1}{{\mu }_x^2+{\mu }_y^2+C_1}$$

(2)

where $C_1$ = 0.01 is a constant used to prevent the denominator from being zero, ${\mu }_x$ and ${\mu }_y$ represent the weighted sum of patches centered at pixel p in image x and image y, respectively. ${\mu }_x$ is defined as follows:

$${\mu }_x=\sum _{i=1}^N{\omega }_i{x}_i$$

(3)

where the weight coefficient ${\omega }_i$ is obtained from a Gaussian function.

Contrast and structural similarity can be represented by a single formula:

$$cs\left(p\right)=\frac{2{\sigma }_{xy}+C_2}{{\sigma }_x^2+{\sigma }_y^2+C_2}$$

(4)

where $C_2$ = 0.03 is also a constant used to prevent the denominator from being zero, ${\sigma }_x^2$, ${\sigma }_y^2$, and ${\sigma }_{xy}$ represent the variance and covariance of patches centered at pixel p in image x and image y. ${\sigma }_x^2$ and ${\sigma }_{xy}$ are defined as:

$${\sigma }_x^2=\sum _{i=1}^N{\omega }_i{(x_i-{\mu }_x)}^2$$

(5)

$${\sigma }_{xy}=\sum _{i=1}^N{\omega }_i(x_i-{\mu }_x)(y_i-{\mu }_y)$$

(6)

We use the index $j=1,\dots ,M$ to represent different Gaussian filters. MS-SSIM is defined as:

$$MS\text{-}SSIM\left(p\right)=l_M\left(p\right)·\prod _{j=1}^{M}c{s}_j\left(p\right)$$

(7)

During the training phase, for an image of size $H\times W$, the MS-SSIM loss can be expressed as follows:

$${\mathcal{L}}_{MS\text{-}SSIM}=\frac{1}{H\times W}\sum _{p}1-MS\text{-}SSIM\left(p\right)$$

(8)

The total loss is calculated by adding the ${\ell }_1$ loss multiplied by the Gaussian filter and the weighted MS-SSIM loss. This formula is shown as Equation (9).

$${\mathcal{L}}_{total}=\alpha ·{\mathcal{L}}_{MS\text{-}SSIM}+(1-\alpha )·G_{{\sigma }_M}·{\mathcal{L}}_{{\ell }_1}$$

(9)

where $\alpha$ represents the weight coefficient.

During the inference stage, we calculated the anomaly localization by computing the MS-SSIM and ${\ell }_1$ error for each pixel.

4. Experiments

4.1. Datasets

4.1.1. MPDD

MPDD [2] is a challenging dataset that focuses on detecting defects in the manufacturing process of painted metal parts. It reflects the real-world situations encountered by human workers on production lines. The dataset includes six categories of metal parts. The images were captured under various spatial orientations, positions, and distance conditions with different light intensities and non-uniform backgrounds. The training set consisted of 888 normal samples, whereas the test set consisted of 176 normal and 282 abnormal samples.

4.1.2. VisA

VisA [3] consists of 10,821 images. There are 9621 normal and 1200 abnormal images. VisA contains 12 subsets, each corresponding to one class of objects. We assigned 90% of the normal images to the training set, whereas 10% of the normal images and all anomalous samples were grouped as the test set.

4.2. Experimental Details

Our study work was implemented in PyTorch using an NVIDIA GeForce GTX 2080. We resized all the original images of the VisA and the MPDD datasets to $256\times 256$ for both training and testing. We divided 20% of the training dataset into validation sets. For each category of these two datasets, we utilized AdamW optimizer [34] with $\beta =(0.5,0.999)$. We set the initial learning rate to ${10}^{-6}$ and used cosine annealing [35] to adjust the learning rate with $T\_max=100$ and $eta\_min={10}^{-6}$. The maximum number of training epochs was set to 500, and the training was stopped early if the loss did not decrease within 20 consecutive epochs.

We evaluated our approach using different metrics for comparison with other baselines. We used the area under the curve (AUC) of the receiver operating characteristic (ROC) to evaluate the performance of image-level anomaly detection and pixel-level anomaly localization.

4.3. Comparative Experiments

4.3.1. MPDD

We compared our proposed method with several state-of-the-art (SOTA) methods on the MPDD dataset, including reconstruction-based methods [12,21] and feature embedding-based methods [7,8,11]. The image-level detection results are listed in

Table 1. Comparison of image-level detection results (AUROC%) on the MPDD dataset.

Method	DAGAN	Skip-GANomaly	PaDiM	CFLOW	PatchCore	Ours
Bracket Black	68.55	61.30	75.60	72.67	81.88	93.42
Bracket Brown	77.07	62.14	85.40	88.84	78.43	93.14
Bracket White	72.11	73.33	82.22	87.78	76.00	89.33
Connector	99.76	73.62	91.67	94.76	96.67	100.00
Metal Plate	85.43	73.24	56.30	99.51	100.00	99.57
Tubes	31.93	46.42	57.51	73.14	59.73	94.16
Avg.	72.48	64.84	74.78	86.12	82.12	94.94

Best results are highlighted in bold.

, and the anomaly segmentation results are presented in

Table 2. Comparison of pixel-level detection results (AUROC%) on the MPDD dataset.

Method	DAGAN	Skip-GANomaly	PaDiM	CFLOW	PatchCore	Ours
Bracket Black	89.73	88.96	94.23	96.88	98.41	98.97
Bracket Brown	81.50	78.07	92.44	97.78	91.46	93.10
Bracket White	70.63	78.81	98.11	98.61	97.44	97.82
Connector	85.73	80.20	97.89	98.39	95.00	98.95
Metal Plate	89.95	89.72	92.93	98.21	96.57	98.78
Tubes	82.31	77.30	93.94	96.43	95.05	99.17
Avg.	83.31	82.19	96.74	97.72	95.66	97.80

Best results are highlighted in bold.

. Experiments demonstrated that our proposed method outperformed previous SOTA methods on the MPDD dataset. The partial visualization results of the proposed method on the MPDD [2] dataset are shown in Figure 4.

Specifically, as shown in Table 1, our proposed method achieved an overall improvement of 8.82% compared to that of the previous best-performing method, CFLOW [11]. The most significant improvement was observed in the tubes that contained multiple instances with a random distribution of positions. These results highlighted the advantages of the proposed method. As shown in Table 2, the best average performance was achieved. However, the proposed method has some limitations. We were unable to achieve satisfactory performance in the brown bracket category. Most defects in the brown bracket category are deformation defects, and our method cannot accurately restore deformations, which hinders the accurate identification of such defects.

4.3.2. VisA

Further, to validate the generalizability and versatility of our method, we compared it with other SOTA methods [7,8,10,11,20,36] on the VisA [3] dataset. The anomaly detection results for the VisA dataset are listed in

Table 3. Comparison of image-level detection results (AUROC%) on the VisA dataset.

Method	DRAEM	RD4AD	PaDiM	CFLOW	FastFlow	PatchCore	Ours
Candle	94.4	92.2	91.6	97.0	92.8	98.6	83.7
Capsules	76.3	90.1	70.7	93.0	71.2	81.6	93.3
Cashew	90.7	99.6	93.0	90.9	91.0	97.3	93.4
Chewing gum	94.2	99.7	98.8	98.3	91.4	99.1	97.7
Fryum	97.4	96.6	88.6	91.1	88.6	96.2	97.3
Macaroni1	95.0	98.4	87.0	69.6	98.3	97.5	91.6
Macaroni2	96.2	97.6	70.5	77.2	86.3	78.1	91.5
PCB1	54.8	97.6	94.7	91.4	77.4	98.5	94.7
PCB2	77.8	91.1	88.5	96.7	61.9	97.3	95.6
PCB3	94.5	95.5	91.0	99.6	74.3	97.9	98.7
PCB4	93.4	96.5	97.5	94.2	80.9	99.6	98.2
Pipe fryum	99.4	97.0	97.0	99.0	72.0	99.8	92.6
Avg.	88.7	96.0	89.1	91.5	82.2	95.1	94.0

Best results are highlighted in bold.

. Experiments demonstrated that our proposed method performed competitively on the VisA dataset.

4.4. Ablation Studies

4.4.1. Effect of $\lambda$

In this study, we employed a noise-to-norm reconstruction paradigm. To validate the effectiveness of adding noise and the effect of the noise coefficient ($\lambda$) on the detection results, we conducted comparative experiments. The results, as shown in

Table 4. Effect of noise coefficient ($\lambda$) on image/pixel-level detection results (AUROC%).

Noise Coefficient	$\mathit{\lambda }=0$	$\mathit{\lambda }=0.2$	$\mathit{\lambda }=0.3$	$\mathit{\lambda }=0.4$
Bracket Black	47.61/77.21	83.71/98.16	93.42/98.97	90.56/99.02
Bracket Brown	82.50/83.93	90.57/91.84	93.14/93.10	84.09/92.86
Bracket White	74.89/89.82	92.78/98.35	89.33/97.82	83.22/96.74
Connector	96.67/93.70	100.00/98.78	100.00/98.95	100.00/98.80
Metal Plate	97.83/97.88	98.92/98.96	99.57/98.78	99.30/98.40
Tubes	39.45/85.88	97.06/98.61	94.16/99.17	93.75/99.44
Avg.	73.16/88.07	93.84/97.45	94.94/97.80	91.82/97.54

Best results are highlighted in bold.

, demonstrate that the overall detection performance was the best when $\lambda =0.3$. Compared to that of the case without added noise ($\lambda =0$), the detection accuracy increased by $22.28\%$, and the segmentation accuracy increased by $9.55\%$. Therefore, we finally set $\lambda =0.3$. These experimental results confirm the significant improvement in anomaly detection achieved using the noise-to-norm reconstruction approach.

4.4.2. Importance of Residual Attention Module

To demonstrate the effectiveness of the proposed residual attention module, we conducted an ablation experiment. In the control group, we replaced the residual attention module with a $1\times 1$ convolutional layer, which was used to change the number of feature channels. The experimental results, as listed in

Table 5. Effect of residual attention module on image/pixel-level detection results (AUROC%).

	Image-Level		Pixel-Level
Residual Attention module	✔	✖	✔	✖
Bracket Black	93.42	50.47	98.97	89.36
Bracket Brown	93.14	75.32	93.10	80.36
Bracket White	92.33	65.92	96.75	85.74
Connector	100.00	62.38	98.95	94.99
Metal Plate	99.57	88.61	98.78	96.69
Tubes	94.16	57.87	99.17	84.93
Avg.	95.44	66.76	97.62	88.68

Best results are highlighted in bold.

, indicate that adding the residual attention module improved the detection accuracy by $28.68\%$ and the segmentation accuracy by $8.94\%$. This demonstrates the significance of incorporating the residual attention module into the model.

5. Conclusions

In this study, an industrial image anomaly detection method based on noise-to-norm reconstruction is proposed. Our proposed method effectively solves the problem of invariant reconstruction of anomalous regions. We enhanced the M-net by incorporating a residual attention module and feature fusion, obtaining a reconstruction network. Our proposed method has significant advantages for handling data with multiple instances and varying object positions. Experimental results demonstrate that our method achieves SOTA performance in anomaly detection and localization on the MPDD dataset, and it also exhibits competitive performance on the VisA dataset.

Our proposed method has limitations in detecting object deformation anomalies. Additionally, when there is noise in the background of certain images in the dataset, it is easy for us to misidentify it as an anomaly. In future work, we will explore methods that combine the feature distribution of positive samples with a reconstruction approach to improve the anomaly detection performance of the model