Image Deblurring Based on Convex NonConvex Sparse Regularization and PlugandPlay Algorithm
Abstract
:1. Introduction
 By incorporating CNC sparse regularization into the image deblurring model, we obtain a new image deblurring model with nonconvex sparse regularization, which can effectively address the limitations of the ${\ell}_{1}$norm. Additionally, we establish the necessary conditions for ensuring the overall convexity of the proposed model.
 After constructing the iterative proximal operator of CNC sparse regularization by using the forwardbackward splitting (FBS) algorithm, we propose an FBS algorithm for the proposed image deblurring model with CNC sparse regularization (FBSCNC) and further derive the corresponding PnPFBSCNC algorithm by substituting the proximal operator with a denoiser.
 The inherent advantages of the proposed algorithm compared with other existing algorithms are verified through numerical experiments.
2. Preliminaries
 (1)
 The proximal operator of ϕ is defined as$${prox}_{\alpha \varphi}\left(x\right)=arg\underset{v}{min}\left\{\varphi \left(v\right)+\frac{1}{2\alpha}{\parallel xv\parallel}_{2}^{2}\right\}.$$
 (2)
 The Moreau envelope of ϕ is defined as$${M}_{\varphi}^{\alpha}\left(x\right)=\underset{v}{min}\left\{\varphi \left(v\right)+\frac{1}{2\alpha}{\parallel xv\parallel}_{2}^{2}\right\}.$$
3. Image Deblurring Model with CNC Sparse Regularization
4. Proposed Algorithms
4.1. The Proximal Operator of CNC Sparse Regularization
4.2. FBSCNC
 1.
 For the datafidelity term:$$\begin{array}{cc}\hfill {v}^{k+1}& ={x}^{k}\alpha \nabla \left(\frac{1}{2}{\parallel y{F}^{1}\mathsf{\Lambda}F{x}^{k}\parallel}_{2}^{2}\right)\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& ={x}^{k}\alpha \left[{F}^{T}{\mathsf{\Lambda}}^{T}{\left({F}^{1}\right)}^{T}\left({F}^{1}\mathsf{\Lambda}F{x}^{k}y\right)\right]\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& ={x}^{k}\alpha \left({F}^{T}{\mathsf{\Lambda}}^{T}\mathsf{\Lambda}F{x}^{k}{F}^{T}{\mathsf{\Lambda}}^{T}Fy\right).\hfill \end{array}$$
 2.
 For the regularization term:$$\begin{array}{cc}\hfill {x}^{k+1}& ={prox}_{\alpha \lambda {\varphi}_{b}}\left({v}^{k+1}\right)\hfill \\ \hfill \phantom{\rule{1.em}{0ex}}& ={prox}_{{\alpha \lambda \parallel \cdot \parallel}_{1}}\left[{v}^{k+1}+\alpha \lambda {b}^{2}\left({x}^{k}{prox}_{\frac{1}{{b}^{2}}{\parallel \cdot \parallel}_{1}}\left({x}^{k}\right)\right)\right].\hfill \end{array}$$
Algorithm 1 FBSCNC for image deblurring 

4.3. PnPFBSCNC
Algorithm 2 PnPFBSCNC for image deblurring 

4.4. Computational Complexity and Convergence Analysis
5. Numerical Experiment
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
 Zhang, K.; Ren, W.; Luo, W.; Lai, W.S.; Stenger, B.; Yang, M.H.; Li, H. Deep image deblurring: A survey. Int. J. Comput. Vis. 2022, 130, 2103–2130. [Google Scholar] [CrossRef]
 Eboli, T.; Sun, J.; Ponce, J. Endtoend interpretable learning of nonblind image deblurring. In Proceedings of the Computer Vision—ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020; Part XVII 16. Springer: Berlin/Heidelberg, Germany, 2020; pp. 314–331. [Google Scholar]
 Yan, Y.; Ren, W.; Guo, Y.; Wang, R.; Cao, X. Image deblurring via extreme channels prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4003–4011. [Google Scholar]
 Vasu, S.; Maligireddy, V.R.; Rajagopalan, A. Nonblind deblurring: Handling kernel uncertainty with CNNs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 3272–3281. [Google Scholar]
 Liu, Y.; Sheng, Z.; Shen, H.L. Guided Image Deblurring by Deep MultiModal Image Fusion. IEEE Access 2022, 10, 130708–130718. [Google Scholar] [CrossRef]
 Zhang, M.; Young, G.S.; Tie, Y.; Gu, X.; Xu, X. A new framework of designing iterative techniques for image deblurring. Pattern Recognit. 2022, 124, 108463. [Google Scholar] [CrossRef]
 Zhang, K.; Li, Y.; Zuo, W.; Zhang, L.; Van Gool, L.; Timofte, R. Plugandplay image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 44, 6360–6376. [Google Scholar] [CrossRef]
 Gavaskar, R.G.; Athalye, C.D.; Chaudhury, K.N. On plugandplay regularization using linear denoisers. IEEE Trans. Image Process. 2021, 30, 4802–4813. [Google Scholar] [CrossRef]
 Gupta, A.; Joshi, N.; Lawrence Zitnick, C.; Cohen, M.; Curless, B. Single image deblurring using motion density functions. In Proceedings of the Computer Vision—ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, 5–11 September 2010; Part I 11. Springer: Berlin/Heidelberg, Germany, 2010; pp. 171–184. [Google Scholar]
 Krishnan, D.; Tay, T.; Fergus, R. Blind deconvolution using a normalized sparsity measure. In Proceedings of the CVPR 2011, Providence, RI, USA, 20–25 June 2011; pp. 233–240. [Google Scholar]
 Xu, L.; Tao, X.; Jia, J. Inverse kernels for fast spatial deconvolution. In Proceedings of the Computer Vision—ECCV 2014: 13th European Conference, Zurich, Switzerland, 6–12 September 2014; Part V 13. Springer: Berlin/Heidelberg, Germany, 2014; pp. 33–48. [Google Scholar]
 Pan, J.; Sun, D.; Pfister, H.; Yang, M.H. Blind image deblurring using dark channel prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 1628–1636. [Google Scholar]
 Hurault, S.; Leclaire, A.; Papadakis, N. Gradient Step Denoiser for convergent PlugandPlay. In Proceedings of the International Conference on Learning Representations (ICLR’22), Virtual Event, 25–29 April 2022. [Google Scholar]
 Fermanian, R.; Pendu, M.L.; Guillemot, C. Learned gradient of a regularizer for plugandplay gradient descent. arXiv 2022, arXiv:2204.13940. [Google Scholar]
 Xu, L.; Zheng, S.; Jia, J. Unnatural L0 Sparse Representation for Natural Image Deblurring. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 1107–1114. [Google Scholar]
 Pan, J.; Hu, Z.; Su, Z.; Yang, M.H. Deblurring Text Images via L0Regularized Intensity and Gradient Prior. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014; pp. 2901–2908. [Google Scholar]
 Levin, A.; Weiss, Y.; Durand, F.; Freeman, W.T. Understanding and evaluating blind deconvolution algorithms. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 1964–1971. [Google Scholar]
 Danielyan, A.; Katkovnik, V.; Egiazarian, K. BM3D frames and variational image deblurring. IEEE Trans. Image Process. 2011, 21, 1715–1728. [Google Scholar] [CrossRef]
 Ye, Z.; Ou, X.; Huang, J.; Chen, Y. Infrared Image Deblurring Based on LpPseudoNorm and HighOrder Overlapping Group Sparsity Regularization. Algorithms 2022, 15, 327. [Google Scholar] [CrossRef]
 Selesnick, I. Sparse regularization via convex analysis. IEEE Trans. Signal Process. 2017, 65, 4481–4494. [Google Scholar] [CrossRef]
 Selesnick, I.; Lanza, A.; Morigi, S.; Sgallari, F. Nonconvex total variation regularization for convex denoising of signals. J. Math. Imaging Vis. 2020, 62, 825–841. [Google Scholar] [CrossRef]
 Lanza, A.; Morigi, S.; Selesnick, I.W.; Sgallari, F. Convex NonConvex Variational Models. In Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging: Mathematical Imaging and Vision; Springer: Berlin/Heidelberg, Germany, 2022; pp. 1–57. [Google Scholar]
 Lanza, A.; Morigi, S.; Selesnick, I.W.; Sgallari, F. Sparsityinducing nonconvex nonseparable regularization for convex image processing. SIAM J. Imaging Sci. 2019, 12, 1099–1134. [Google Scholar] [CrossRef]
 Teodoro, A.M.; BioucasDias, J.M.; Figueiredo, M.A. Image restoration and reconstruction using variable splitting and classadapted image priors. In Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2016; pp. 3518–3522. [Google Scholar]
 Kupyn, O.; Budzan, V.; Mykhailych, M.; Mishkin, D.; Matas, J. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 8183–8192. [Google Scholar]
 Li, Y.; Tofighi, M.; Monga, V.; Eldar, Y.C. An algorithm unrolling approach to deep image deblurring. In Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019; pp. 7675–7679. [Google Scholar]
 Mittal, T.; Agrawal, P.; Pahwa, E.; Makwana, A. NFResNet: Multiscale and Ushaped Networks for Deblurring. arXiv 2022, arXiv:2212.05909. [Google Scholar]
 Tao, X.; Gao, H.; Shen, X.; Wang, J.; Jia, J. Scalerecurrent network for deep image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 8174–8182. [Google Scholar]
 Liang, C.H.; Chen, Y.A.; Liu, Y.C.; Hsu, W.H. Raw image deblurring. IEEE Trans. Multimed. 2020, 24, 61–72. [Google Scholar] [CrossRef]
 Zou, W.; Jiang, M.; Zhang, Y.; Chen, L.; Lu, Z.; Wu, Y. Sdwnet: A straight dilated network with wavelet transformation for image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021; pp. 1895–1904. [Google Scholar]
 Tomosada, H.; Kudo, T.; Fujisawa, T.; Ikehara, M. GANbased image deblurring using DCT loss with customized datasets. IEEE Access 2021, 9, 135224–135233. [Google Scholar] [CrossRef]
 Chen, T.; Chen, X.; Chen, W.; Wang, Z.; Heaton, H.; Liu, J.; Yin, W. Learning to optimize: A primer and a benchmark. J. Mach. Learn. Res. 2022, 23, 8562–8620. [Google Scholar]
 Mukherjee, S.; Hauptmann, A.; Öktem, O.; Pereyra, M.; Schönlieb, C.B. Learned Reconstruction Methods with Convergence Guarantees: A survey of concepts and applications. IEEE Signal Process. Mag. 2023, 40, 164–182. [Google Scholar] [CrossRef]
 Shlezinger, N.; Whang, J.; Eldar, Y.C.; Dimakis, A.G. ModelBased Deep Learning. Proc. IEEE 2023, 111, 465–499. [Google Scholar] [CrossRef]
 Ryu, E.; Liu, J.; Wang, S.; Chen, X.; Wang, Z.; Yin, W. Plugandplay methods provably converge with properly trained denoisers. In Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019; pp. 5546–5557. [Google Scholar]
 Zhang, K.; Zuo, W.; Gu, S.; Zhang, L. Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 3929–3938. [Google Scholar]
 Nair, P.; Chaudhury, K.N. On the Construction of Averaged Deep Denoisers for Image Regularization. arXiv 2022, arXiv:2207.07321. [Google Scholar]
 Kamilov, U.S.; Bouman, C.A.; Buzzard, G.T.; Wohlberg, B. Plugandplay methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications. IEEE Signal Process. Mag. 2023, 40, 85–97. [Google Scholar] [CrossRef]
 Li, J.; Li, J.; Xie, Z.; Zou, J. Plugandplay ADMM for MRI reconstruction with convex nonconvex sparse regularization. IEEE Access 2021, 9, 148315–148324. [Google Scholar] [CrossRef]
 Xu, Y.; Qu, M.; Liu, L.; Liu, G.; Zou, J. Plugandplay algorithms for convex nonconvex regularization: Convergence analysis and applications. Math. Methods Appl. Sci. 2023. [CrossRef]
 Rudin, L.I.; Osher, S.; Fatemi, E. Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 1992, 60, 259–268. [Google Scholar] [CrossRef]
 Dabov, K.; Foi, A.; Katkovnik, V.; Egiazarian, K. Image denoising by sparse 3D transformdomain collaborative filtering. IEEE Trans. Image Process. 2007, 16, 2080–2095. [Google Scholar] [CrossRef]
 Selesnick, I. Total variation denoising via the Moreau envelope. IEEE Signal Process. Lett. 2017, 24, 216–220. [Google Scholar] [CrossRef]
 Perelli, A.; Lexa, M.; Can, A.; Davies, M.E. Compressive computed tomography reconstruction through denoising approximate message passing. SIAM J. Imaging Sci. 2020, 13, 1860–1897. [Google Scholar] [CrossRef]
 Zhao, X.L.; Xu, W.H.; Jiang, T.X.; Wang, Y.; Ng, M.K. Deep plugandplay prior for lowrank tensor completion. Neurocomputing 2020, 400, 137–149. [Google Scholar] [CrossRef]
 Liu, T.; Yin, Q.; Yang, J.; Wang, Y.; An, W. Combining Deep Denoiser and Lowrank Priors for Infrared Small Target Detection. Pattern Recognit. 2023, 135, 109184. [Google Scholar] [CrossRef]
 Martin, D.; Fowlkes, C.; Tal, D.; Malik, J. A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In Proceedings of the 8th International Conference on Computer Vision, Vancouver, BC, Canada, 7–14 July 2001; Volume 2, pp. 416–423. [Google Scholar]
$\mathit{\nu}$  Method  CPU Time  Blur kernel  

1  2  3  4  5  6  7  8  
0.01  IRCNN  24.32  32.68  32.20  31.35  32.17  32.24  32.65  31.46  31.42 
DPIR  25.93  33.61  33.18  32.95  33.01  34.03  34.29  33.02  32.70  
GSPnP  26.60  33.47  32.97  32.83  32.79  34.01  34.19  32.88  32.49  
PnPFBSCNC  24.28  33.98  33.56  33.37  33.45  34.48  34.69  33.38  33.16  
0.03  IRCNN  27.58  29.25  28.81  28.84  28.56  29.72  29.61  28.82  28.49 
DPIR  28.15  29.31  28.98  29.13  28.68  30.14  30.17  29.25  28.82  
GSPnP  30.57  29.16  28.82  29.13  28.54  30.26  30.15  29.26  28.86  
PnPFBSCNC  28.07  29.81  29. 49  29.68  29.25  30.67  30.66  29.77  29.37  
0.05  IRCNN  30.24  26.92  26.67  27.16  26.29  28.22  28.00  27.14  26.75 
DPIR  31.52  27.45  27.26  27.67  26.91  28.55  28.27  27.71  27.23  
GSPnP  32.67  27.38  27.21  27.63  26.92  28.63  28.36  27.74  27.33  
PnPFBSCNC  31.19  28.03  27.76  28.22  27.51  29.12  28.95  28.24  27.84 
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. 
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, Y.; Xu, Y.; Li, T.; Zhang, T.; Zou, J. Image Deblurring Based on Convex NonConvex Sparse Regularization and PlugandPlay Algorithm. Algorithms 2023, 16, 574. https://doi.org/10.3390/a16120574
Wang Y, Xu Y, Li T, Zhang T, Zou J. Image Deblurring Based on Convex NonConvex Sparse Regularization and PlugandPlay Algorithm. Algorithms. 2023; 16(12):574. https://doi.org/10.3390/a16120574
Chicago/Turabian StyleWang, Yi, Yating Xu, Tianjian Li, Tao Zhang, and Jian Zou. 2023. "Image Deblurring Based on Convex NonConvex Sparse Regularization and PlugandPlay Algorithm" Algorithms 16, no. 12: 574. https://doi.org/10.3390/a16120574