# Image Denoising by Deep Convolution Based on Sparse Representation

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

#### 2.1. Sparse-Based

#### 2.2. Deep Learning-Based Denoising

## 3. Proposed Framework

#### 3.1. Local Prior for Image Blocks

#### 3.2. Global Prior for Image Blocks

#### 3.3. Patch Denoiser

#### 3.3.1. Sparse Coding

#### 3.3.2. $\lambda $ Estimation

#### 3.3.3. Patch Reconstruction

#### 3.4. Patch Averaging

#### 3.5. Loss Function

## 4. Simulation Results

#### 4.1. Simulation Experiments

#### 4.2. Analysis of Simulation Results

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

AWGN | additive white Gaussian noise; |

ISTA | iterative shrinkage thresholding algorithm |

PSNR | signal-to-noise ratio |

BM3D | block-matching and 3D filtering |

MAP | maximum a posteriori |

WNNM | weighted nuclear norm minimization |

TNRD | linear reaction diffusion |

BN | batch normalization |

DCT | discrete cosine transform |

## References

- Lei, Y.; Yuan, W.; Wang, H.; Wenhu, Y.; Bo, W. A skin segmentation algorithm based on stacked autoencoders. IEEE Trans. Multimed.
**2016**, 19, 740–749. [Google Scholar] [CrossRef] - Fonseca, L.M.G.; Namikawa, L.M.; Castejon, E.F. Digital image processing in remote sensing. In Proceedings of the 2009 Tutorials of the XXII Brazilian Symposium on Computer Graphics and Image Processing, Rio de Janeiro, Brazil, 11–14 October 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 59–71. [Google Scholar]
- Wang, G.; Ye, J.C.; De Man, B. Deep learning for tomographic image reconstruction. Nat. Mach. Intell.
**2020**, 2, 737–748. [Google Scholar] [CrossRef] - Schermelleh, L.; Ferrand, A.; Huser, T.; Eggeling, C.; Sauer, M.; Biehlmaier, O.; Drummen, G.P. Super-resolution microscopy demystified. Nat. Cell Biol.
**2019**, 21, 72–84. [Google Scholar] [CrossRef] [PubMed] - Ali, A.M.; Benjdira, B.; Koubaa, A.; El-Shafai, W.; Khan, Z.; Boulila, W. Vision Transformers in Image Restoration: A Survey. Sensors
**2023**, 23, 2385. [Google Scholar] [CrossRef] [PubMed] - Unni, V.; Ghosh, S.; Chaudhury, K.N. Linearized ADMM and fast nonlocal denoising for efficient plug-and-play restoration. In Proceedings of the 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Anaheim, CA, USA, 26–29 November 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 11–15. [Google Scholar]
- Aharon, M.; Elad, M.; Bruckstein, A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process.
**2006**, 54, 4311–4322. [Google Scholar] [CrossRef] - Dabov, K.; Foi, A.; Katkovnik, V.; Egiazarian, K. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process.
**2007**, 16, 2080–2095. [Google Scholar] - Sun, D.; Gao, Q.; Lu, Y.; Huang, Z.; Li, T. A novel image denoising algorithm using linear Bayesian MAP estimation based on sparse representation. Signal Process.
**2014**, 100, 132–145. [Google Scholar] [CrossRef] - Xu, H.; Jia, X.; Cheng, L.; Huang, H. Affine non-local Bayesian image denoising algorithm. Vis. Comput.
**2023**, 39, 99–118. [Google Scholar] [CrossRef] - Glynn, C.; Tokdar, S.T.; Howard, B.; Banks, D.L. Bayesian analysis of dynamic linear topic models. Bayesian Anal.
**2019**, 14, 53–80. [Google Scholar] [CrossRef] - Kollem, S.; Reddy, K.R.L.; Rao, D.S. A review of image denoising and segmentation methods based on medical images. Int. J. Mach. Learn. Comput.
**2019**, 9, 288–295. [Google Scholar] [CrossRef] - de Ridder, D.; Duin, R.P.; Verbeek, P.W.; Van Vliet, L. The applicability of neural networks to non-linear image processing. Pattern Anal. Appl.
**1999**, 2, 111–128. [Google Scholar] [CrossRef] - Elad, M.; Aharon, M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process.
**2006**, 15, 3736–3745. [Google Scholar] [CrossRef] [PubMed] - Osher, S.; Burger, M.; Goldfarb, D.; Xu, J.; Yin, W. An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul.
**2005**, 4, 460–489. [Google Scholar] [CrossRef] - Ren, D.; Zuo, W.; Zhang, D.; Zhang, L.; Yang, M.H. Simultaneous fidelity and regularization learning for image restoration. IEEE Trans. Pattern Anal. Mach. Intell.
**2019**, 43, 284–299. [Google Scholar] [CrossRef] - Cross, G.R.; Jain, A.K. Markov random field texture models. IEEE Trans. Pattern Anal. Mach. Intell.
**1983**, PAMI-5, 25–39. [Google Scholar] [CrossRef] - Gu, S.; Zhang, L.; Zuo, W.; Feng, X. Weighted nuclear norm minimization with application to image denoising. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 2862–2869. [Google Scholar]
- Mairal, J.; Bach, F.; Ponce, J.; Sapiro, G.; Zisserman, A. Non-local sparse models for image restoration. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, St. Louis, MO, USA, 4–7 October 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 2272–2279. [Google Scholar]
- Schmidt, U.; Roth, S. Shrinkage fields for effective image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 2774–2781. [Google Scholar]
- Chen, Y.; Pock, T. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell.
**2016**, 39, 1256–1272. [Google Scholar] [CrossRef] - Zuo, W.; Zhang, L.; Song, C.; Zhang, D.; Gao, H. Gradient histogram estimation and preservation for texture enhanced image denoising. IEEE Trans. Image Process.
**2014**, 23, 2459–2472. [Google Scholar] - Khmag, A.; Ramli, A.R.; Kamarudin, N. Clustering-based natural image denoising using dictionary learning approach in wavelet domain. Soft Comput.
**2019**, 23, 8013–8027. [Google Scholar] [CrossRef] - Tallapragada, V.; Manga, N.A.; Kumar, G.; Naresh, M.V. Mixed image denoising using weighted coding and non-local similarity. SN Appl. Sci.
**2020**, 2, 997. [Google Scholar] [CrossRef] - Mahdaoui, A.E.; Ouahabi, A.; Moulay, M.S. Image denoising using a compressive sensing approach based on regularization constraints. Sensors
**2022**, 22, 2199. [Google Scholar] [CrossRef] - Fukushima, K.; Miyake, S. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In Competition and Cooperation in Neural Nets; Springer: Berlin/Heidelberg, Germany, 1982; pp. 267–285. [Google Scholar]
- Zhou, Y.; Chellappa, R.; Jenkins, B. A novel approach to image restoration based on a neural network. In Proceedings of the International Conference on Neural Networks, San Diego, CA, USA, 21 June 1987. [Google Scholar]
- Tamura, S. An analysis of a noise reduction neural network. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, Glasgow, UK, 23–26 May 1989; IEEE: Piscataway, NJ, USA, 1989; pp. 2001–2004. [Google Scholar]
- Liang, J.; Liu, R. Stacked denoising autoencoder and dropout together to prevent overfitting in deep neural network. In Proceedings of the 2015 8th International Congress on Image and Signal Processing (CISP), Shenyang, China, 14–16 October 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 697–701. [Google Scholar]
- Zhang, Z.; Wang, L.; Kai, A.; Yamada, T.; Li, W.; Iwahashi, M. Deep neural network-based bottleneck feature and denoising autoencoder-based dereverberation for distant-talking speaker identification. EURASIP J. Audio, Speech, Music. Process.
**2015**, 2015, 1–13. [Google Scholar] [CrossRef] - Yuan, D.; Fan, N.; He, Z. Learning target-focusing convolutional regression model for visual object tracking. Knowl.-Based Syst.
**2020**, 194, 105526. [Google Scholar] [CrossRef] - Tian, C.; Xu, Y.; Zuo, W.; Zhang, B.; Fei, L.; Lin, C.W. Coarse-to-fine CNN for image super-resolution. IEEE Trans. Multimed.
**2020**, 23, 1489–1502. [Google Scholar] [CrossRef] - Burger, H.C.; Schuler, C.J.; Harmeling, S. Image denoising: Can plain neural networks compete with BM3D? In Proceedings of the 2012 IEEE conference on computer vision and pattern recognition, Providence, RI, USA, 16–21 June 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 2392–2399. [Google Scholar]
- Mao, X.; Shen, C.; Yang, Y.B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Adv. Neural Inf. Process. Syst.
**2016**, 29, 1–9. [Google Scholar] - Zhang, K.; Zuo, W.; Chen, Y.; Meng, D.; Zhang, L. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans. Image Process.
**2017**, 26, 3142–3155. [Google Scholar] [CrossRef] - Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning (PMLR), Lille, France, 7–9 July 2015; pp. 448–456. [Google Scholar]
- Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June 2010. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Phoenix, AZ, USA, 25–28 September 2016; pp. 770–778. [Google Scholar]
- Ghose, S.; Singh, N.; Singh, P. Image denoising using deep learning: Convolutional neural network. In Proceedings of the 2020 10th International Conference on Cloud Computing, Data Science and Engineering (Confluence), Noida, India, 29–31 January 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 511–517. [Google Scholar]
- Gai, S.; Bao, Z. New image denoising algorithm via improved deep convolutional neural network with perceptive loss. Expert Syst. Appl.
**2019**, 138, 112815. [Google Scholar] [CrossRef] - Zhang, K.; Zuo, W.; Zhang, L. Deep plug-and-play super-resolution for arbitrary blur kernels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–17 June 2019; pp. 1671–1681. [Google Scholar]
- Cheng, S.; Wang, Y.; Huang, H.; Liu, D.; Fan, H.; Liu, S. Nbnet: Noise basis learning for image denoising with subspace projection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 4896–4906. [Google Scholar]
- Zhang, K.; Zuo, W.; Zhang, L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process.
**2018**, 27, 4608–4622. [Google Scholar] [CrossRef] - Guo, S.; Yan, Z.; Zhang, K.; Zuo, W.; Zhang, L. Toward convolutional blind denoising of real photographs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–17 June 2019; pp. 1712–1722. [Google Scholar]
- Gurrola-Ramos, J.; Dalmau, O.; Alarcón, T.E. A residual dense U-Net neural network for image denoising. IEEE Access
**2021**, 9, 31742–31754. [Google Scholar] [CrossRef] - Jia, F.; Wong, W.H.; Zeng, T. DDUNet: Dense dense U-net with applications in image denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 354–364. [Google Scholar]
- Beck, A.; Teboulle, M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci.
**2009**, 2, 183–202. [Google Scholar] [CrossRef]

**Figure 4.**Denoising results of the different methods on an image from BSD68 with $\delta $ = 25. Original image, WNNM/30.42 dB, DnCNN/30.75 dB, BM3D/30.33 dB, MLP/30.47 dB, Ours/30.78 dB.

**Figure 5.**Denoising results of the different methods on an image from BSD68 with $\delta $ = 25. Original image, WNNM/29.76 dB, DnCNN/30.19 dB, BM3D/29.53 dB, MLP/29.72 dB, Ours/30.17 dB.

**Figure 6.**Denoising results of the different methods on an image from BSD68 with $\delta $ = 25.Original image, WNNM/27.30 dB, DnCNN/27.68 dB, BM3D/27.11 dB, MLP/27.51 dB, Ours/27.72 dB.

**Figure 7.**Denoising results of the different methods on an image from BSD68 with $\delta $ = 25. Original image, WNNM/37.20 dB, DnCNN/38.41 dB, BM3D/36.94 dB, MLP/37.53 dB, Ours/38.36 dB.

**Figure 8.**Denoising results of the different methods on an image from BSD68 with $\delta $ = 25. Original image, WNNM/32.56 dB, DnCNN/33.05 dB, BM3D/32.41 dB, MLP/29.72 dB, Ours/33.09 dB.

**Table 1.**Average PSNR (dB) of the different methods on BSD68 with different noise levels of 15, 25, and 50.

Method | $\delta $ = 15 | $\delta $ = 25 | $\delta $ = 50 |
---|---|---|---|

BM3D | 31.07 | 28.57 | 25.62 |

KSVD | 30.91 | 28.32 | 25.03 |

WNNM | 31.37 | 28.83 | 25.87 |

MLP | - | 28.96 | 26.03 |

TNRD | 31.42 | 28.92 | 25.97 |

DnCNN | 31.72 | 29.19 | 26.23 |

Ours | 31.74 | 29.22 | 26.24 |

**Table 2.**Average SSIM (dB) of the different methods on BSD68 with different noise levels of 15, 25, and 50.

Method | $\delta $ = 15 | $\delta $ = 25 | $\delta $ = 50 |
---|---|---|---|

BM3D | 0.8717 | 0.8013 | 0.6864 |

KSVD | 0.8692 | 0.7876 | 0.6322 |

WNNM | 0.8766 | 0.8074 | 0.6984 |

TNRD | 0.8813 | 0.8157 | 0.7029 |

DnCNN | 0.8826 | 0.8186 | 0.7072 |

Ours | 0.8837 | 0.8199 | 0.7088 |

**Table 3.**Average PSNR (dB) of the different methods on Set12 with different noise levels of 15, 25, and 50.

Method | $\delta $ = 15 | $\delta $ = 25 | $\delta $ = 50 |
---|---|---|---|

BM3D | 32.37 | 29.97 | 26.72 |

KSVD | 31.75 | 29.17 | 25.58 |

WNNM | 32.70 | 30.26 | 27.04 |

MLP | - | 30.03 | 26.78 |

DnCNN | 32.86 | 30.43 | 27.18 |

Ours | 32.89 | 30.45 | 27.16 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Bian, S.; He, X.; Xu, Z.; Zhang, L.
Image Denoising by Deep Convolution Based on Sparse Representation. *Computers* **2023**, *12*, 112.
https://doi.org/10.3390/computers12060112

**AMA Style**

Bian S, He X, Xu Z, Zhang L.
Image Denoising by Deep Convolution Based on Sparse Representation. *Computers*. 2023; 12(6):112.
https://doi.org/10.3390/computers12060112

**Chicago/Turabian Style**

Bian, Shengqin, Xinyu He, Zhengguang Xu, and Lixin Zhang.
2023. "Image Denoising by Deep Convolution Based on Sparse Representation" *Computers* 12, no. 6: 112.
https://doi.org/10.3390/computers12060112