# Pixel Super-Resolution Phase Retrieval for Lensless On-Chip Microscopy via Accelerated Wirtinger Flow

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Problem Formulation

#### 2.1. Forward Model

#### 2.2. Regularized Inversion

## 3. Derivation of Algorithms

#### 3.1. Accelerated Wirtinger Flow

#### 3.2. Convergence Analysis

**Convergence**

**Theorem**

**1.**

## 4. Experimental Results

#### 4.1. System Configuration

#### 4.2. Simulation Studies

#### 4.3. Optical Experiments

## 5. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Huang, Z.; Cao, L. High bandwidth-utilization digital holographic multiplexing: An approach using Kramers–Kronig relations. Adv. Photonics Res.
**2022**, 3, 2100273. [Google Scholar] [CrossRef] - Pan, A.; Zuo, C.; Yao, B. High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine. Rep. Prog. Phys.
**2020**, 83, 096101. [Google Scholar] [CrossRef] [PubMed] - Park, J.; Brady, D.J.; Zheng, G.; Tian, L.; Gao, L. Review of bio-optical imaging systems with a high space-bandwidth product. Adv. Photonics
**2021**, 3, 044001. [Google Scholar] [CrossRef] - Greenbaum, A.; Luo, W.; Khademhosseinieh, B.; Su, T.W.; Coskun, A.F.; Ozcan, A. Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy. Sci. Rep.
**2013**, 3, 1717. [Google Scholar] [CrossRef] - Fan, Y.; Li, J.; Lu, L.; Sun, J.; Hu, Y.; Zhang, J.; Li, Z.; Shen, Q.; Wang, B.; Zhang, R.; et al. Smart computational light microscopes (SCLMs) of smart computational imaging laboratory (SCILab). PhotoniX
**2021**, 2, 19. [Google Scholar] [CrossRef] - Ozcan, A.; McLeod, E. Lensless imaging and sensing. Annu. Rev. Biomed. Eng.
**2016**, 18, 77–102. [Google Scholar] [CrossRef] [Green Version] - Jiang, S.; Guo, C.; Song, P.; Zhou, N.; Bian, Z.; Zhu, J.; Wang, R.; Dong, P.; Zhang, Z.; Liao, J.; et al. Resolution-enhanced parallel coded ptychography for high-throughput optical imaging. ACS Photonics
**2021**, 8, 3261–3271. [Google Scholar] [CrossRef] - Sobieranski, A.C.; Inci, F.; Tekin, H.C.; Yuksekkaya, M.; Comunello, E.; Cobra, D.; Von Wangenheim, A.; Demirci, U. Portable lensless wide-field microscopy imaging platform based on digital inline holography and multi-frame pixel super-resolution. Light Sci. Appl.
**2015**, 4, e346. [Google Scholar] [CrossRef] [Green Version] - Guo, C.; Liu, X.; Zhang, F.; Du, Y.; Zheng, S.; Wang, Z.; Zhang, X.; Kan, X.; Liu, Z.; Wang, W. Lensfree on-chip microscopy based on single-plane phase retrieval. Opt. Express
**2022**, 30, 19855–19870. [Google Scholar] [CrossRef] - Park, Y.; Depeursinge, C.; Popescu, G. Quantitative phase imaging in biomedicine. Nat. Photonics
**2018**, 12, 578–589. [Google Scholar] [CrossRef] - Lee, K.; Kim, K.; Jung, J.; Heo, J.; Cho, S.; Lee, S.; Chang, G.; Jo, Y.; Park, H.; Park, Y. Quantitative phase imaging techniques for the study of cell pathophysiology: From principles to applications. Sensors
**2013**, 13, 4170–4191. [Google Scholar] [CrossRef] [PubMed] - Bettenworth, D.; Bokemeyer, A.; Poremba, C.; Ding, N.S.; Ketelhut, S.; Lenz, P.; Kemper, B. Quantitative phase microscopy for evaluation of intestinal inflammation and wound healing utilizing label-free biophysical markers. Histol. Histopathol.
**2018**, 33, 417–432. [Google Scholar] [PubMed] - Yoon, J.; Jo, Y.; Kim, M.h.; Kim, K.; Lee, S.; Kang, S.J.; Park, Y. Identification of non-activated lymphocytes using three-dimensional refractive index tomography and machine learning. Sci. Rep.
**2017**, 7, 1–10. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Marquet, P.; Depeursinge, C.; Magistretti, P.J. Review of quantitative phase-digital holographic microscopy: Promising novel imaging technique to resolve neuronal network activity and identify cellular biomarkers of psychiatric disorders. Neurophotonics
**2014**, 1, 020901. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Kemper, B.; Carl, D.D.; Schnekenburger, J.; Bredebusch, I.; Schäfer, M.; Domschke, W.; von Bally, G. Investigation of living pancreas tumor cells by digital holographic microscopy. J. Biomed. Opt.
**2006**, 11, 034005. [Google Scholar] [CrossRef] - Shechtman, Y.; Eldar, Y.C.; Cohen, O.; Chapman, H.N.; Miao, J.; Segev, M. Phase retrieval with application to optical imaging: A contemporary overview. IEEE Signal Process. Mag.
**2015**, 32, 87–109. [Google Scholar] [CrossRef] [Green Version] - Mudanyali, O.; McLeod, E.; Luo, W.; Greenbaum, A.; Coskun, A.F.; Hennequin, Y.; Allier, C.P.; Ozcan, A. Wide-field optical detection of nanoparticles using on-chip microscopy and self-assembled nanolenses. Nat. Photonics
**2013**, 7, 247–254. [Google Scholar] [CrossRef] [Green Version] - Zhang, Y.; Pedrini, G.; Osten, W.; Tiziani, H.J. Whole optical wave field reconstruction from double or multi in-line holograms by phase retrieval algorithm. Opt. Express
**2003**, 11, 3234–3241. [Google Scholar] [CrossRef] - Faulkner, H.M.L.; Rodenburg, J.M. Movable aperture lensless transmission microscopy: A novel phase retrieval algorithm. Phys. Rev. Lett.
**2004**, 93, 023903. [Google Scholar] [CrossRef] [Green Version] - Zhang, F.; Pedrini, G.; Osten, W. Phase retrieval of arbitrary complex-valued fields through aperture-plane modulation. Phys. Rev. A
**2007**, 75, 043805. [Google Scholar] [CrossRef] - Bao, P.; Zhang, F.; Pedrini, G.; Osten, W. Phase retrieval using multiple illumination wavelengths. Opt. Lett.
**2008**, 33, 309–311. [Google Scholar] [CrossRef] - Gerchberg, R.W.; Saxton, W. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik
**1971**, 35, 237–250. [Google Scholar] - Fienup, J.R. Phase retrieval algorithms: A comparison. Appl. Opt.
**1982**, 21, 2758–2769. [Google Scholar] [CrossRef] [Green Version] - Candès, E.J.; Li, X.; Soltanolkotabi, M. Phase retrieval via Wirtinger flow: Theory and algorithms. IEEE Trans. Inf. Theory
**2015**, 61, 1985–2007. [Google Scholar] [CrossRef] [Green Version] - Greenbaum, A.; Feizi, A.; Akbari, N.; Ozcan, A. Wide-field computational color imaging using pixel super-resolved on-chip microscopy. Opt. Express
**2013**, 21, 12469–12483. [Google Scholar] [CrossRef] [Green Version] - Bishara, W.; Su, T.W.; Coskun, A.F.; Ozcan, A. Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution. Opt. Express
**2010**, 18, 11181–11191. [Google Scholar] [CrossRef] - Greenbaum, A.; Ozcan, A. Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy. Opt. Express
**2012**, 20, 3129–3143. [Google Scholar] [CrossRef] - Luo, W.; Greenbaum, A.; Zhang, Y.; Ozcan, A. Synthetic aperture-based on-chip microscopy. Light Sci. Appl.
**2015**, 4, e261. [Google Scholar] [CrossRef] - Fournier, C.; Jolivet, F.; Denis, L.; Verrier, N.; Thiebaut, E.; Allier, C.; Fournel, T. Pixel super-resolution in digital holography by regularized reconstruction. Appl. Opt.
**2017**, 56, 69–77. [Google Scholar] [CrossRef] [Green Version] - Lee, H.; Kim, J.; Kim, J.; Jeon, P.; Lee, S.A.; Kim, D. Noniterative sub-pixel shifting super-resolution lensless digital holography. Opt. Express
**2021**, 29, 29996–30006. [Google Scholar] [CrossRef] - Luo, W.; Zhang, Y.; Göröcs, Z.; Feizi, A.; Ozcan, A. Propagation phasor approach for holographic image reconstruction. Sci. Rep.
**2016**, 6, 22738. [Google Scholar] [CrossRef] - Zhang, J.; Sun, J.; Chen, Q.; Zuo, C. Resolution analysis in a lens-free on-chip digital holographic microscope. IEEE Trans. Comput. Imaging
**2020**, 6, 697–710. [Google Scholar] [CrossRef] [Green Version] - Gao, Y.; Cao, L. Generalized optimization framework for pixel super-resolution imaging in digital holography. Opt. Express
**2021**, 29, 28805–28823. [Google Scholar] [CrossRef] - Shen, C.; Guo, C.; Geng, Y.; Tan, J.; Liu, S.; Liu, Z. Noise-robust pixel-super-resolved multi-image phase retrieval with coherent illumination. J. Opt.
**2018**, 20, 115703. [Google Scholar] [CrossRef] - Guo, C.; Zhang, F.; Zhang, X.; Kan, X.; Tan, J.; Liu, S.; Liu, Z. Lensfree super-resolved imaging based on adaptive Wiener filter and guided phase retrieval algorithm. J. Opt.
**2020**, 22, 055703. [Google Scholar] [CrossRef] - Jiang, S.; Guo, C.; Hu, P.; Hu, D.; Song, P.; Wang, T.; Bian, Z.; Zhang, Z.; Zheng, G. High-throughput lensless whole slide imaging via continuous height-varying modulation of a tilted sensor. Opt. Lett.
**2021**, 46, 5212–5215. [Google Scholar] [CrossRef] - Luo, W.; Zhang, Y.; Feizi, A.; Göröcs, Z.; Ozcan, A. Pixel super-resolution using wavelength scanning. Light Sci. Appl.
**2016**, 5, e16060. [Google Scholar] [CrossRef] [Green Version] - Song, P.; Wang, R.; Zhu, J.; Wang, T.; Bian, Z.; Zhang, Z.; Hoshino, K.; Murphy, M.; Jiang, S.; Guo, C.; et al. Super-resolved multispectral lensless microscopy via angle-tilted, wavelength-multiplexed ptychographic modulation. Opt. Lett.
**2020**, 45, 3486–3489. [Google Scholar] [CrossRef] - Wu, X.; Sun, J.; Zhang, J.; Lu, L.; Chen, R.; Chen, Q.; Zuo, C. Wavelength-scanning lensfree on-chip microscopy for wide-field pixel-super-resolved quantitative phase imaging. Opt. Lett.
**2021**, 46, 2023–2026. [Google Scholar] [CrossRef] - Wang, Q.; Ma, J.; Su, P. A multi-wavelength phase retrieval with multi-strategy for lensfree onchip holography. Front. Photonics
**2022**, 3, 7. [Google Scholar] [CrossRef] - Gao, Y.; Cao, L. High-fidelity pixel-super-resolved complex field reconstruction via adaptive smoothing. Opt. Lett.
**2020**, 45, 6807–6810. [Google Scholar] [CrossRef] - Katkovnik, V.; Shevkunov, I.; Petrov, N.V.; Egiazarian, K. Computational super-resolution phase retrieval from multiple phase-coded diffraction patterns: Simulation study and experiments. Optica
**2017**, 4, 786–794. [Google Scholar] [CrossRef] - Guo, Y.; Guo, R.; Qi, P.; Zhou, Y.; Zhang, Z.; Zheng, G.; Zhong, J. Robust multi-angle structured illumination lensless microscopy via illumination angle calibration. Opt. Lett.
**2022**, 47, 1847–1850. [Google Scholar] [CrossRef] - Zhang, H.; Bian, Z.; Jiang, S.; Liu, J.; Song, P.; Zheng, G. Field-portable quantitative lensless microscopy based on translated speckle illumination and sub-sampled ptychographic phase retrieval. Opt. Lett.
**2019**, 44, 1976–1979. [Google Scholar] [CrossRef] - Jiang, S.; Zhu, J.; Song, P.; Guo, C.; Bian, Z.; Wang, R.; Huang, Y.; Wang, S.; Zhang, H.; Zheng, G. Wide-field, high-resolution lensless on-chip microscopy via near-field blind ptychographic modulation. Lab Chip
**2020**, 20, 1058–1065. [Google Scholar] [CrossRef] [Green Version] - Lv, W.; Zhang, J.; Chen, H.; Yang, D.; Ruan, T.; Zhu, Y.; Tao, Y.; Shi, Y. Resolution-enhanced ptychography framework with an equivalent upsampling and precise position. Appl. Opt.
**2022**, 61, 2903–2909. [Google Scholar] [CrossRef] - Zhang, J.; Sun, J.; Chen, Q.; Li, J.; Zuo, C. Adaptive pixel-super-resolved lensfree in-line digital holography for wide-field on-chip microscopy. Sci. Rep.
**2017**, 7, 1–15. [Google Scholar] [CrossRef] - Rivenson, Y.; Wu, Y.; Wang, H.; Zhang, Y.; Feizi, A.; Ozcan, A. Sparsity-based multi-height phase recovery in holographic microscopy. Sci. Rep.
**2016**, 6, 1–9. [Google Scholar] [CrossRef] [Green Version] - Zhang, F.; Guo, C.; Zhai, Y.; Tan, J.; Liu, S.; Tan, C.; Chen, H.; Liu, Z. A noise-robust multi-intensity phase retrieval method based on structural patch decomposition. J. Opt.
**2020**, 22, 075706. [Google Scholar] [CrossRef] - Shimobaba, T.; Sato, Y.; Miura, J.; Takenouchi, M.; Ito, T. Real-time digital holographic microscopy using the graphic processing unit. Opt. Express
**2008**, 16, 11776–11781. [Google Scholar] [CrossRef] - Tian, L.; Li, X.; Ramchandran, K.; Waller, L. Multiplexed coded illumination for Fourier ptychography with an LED array microscope. Biomed. Opt. Express
**2014**, 5, 2376–2389. [Google Scholar] [CrossRef] [Green Version] - Edgar, M.P.; Gibson, G.M.; Padgett, M.J. Principles and prospects for single-pixel imaging. Nat. Photonics
**2019**, 13, 13–20. [Google Scholar] [CrossRef] - Zhang, H.; Chi, Y.; Liang, Y. Provable non-convex phase retrieval with outliers: Median truncated Wirtinger flow. In Proceedings of the 33rd International Conference on Machine Learning, PMLR 48, New York, NY, USA, 20–22 June 2016; pp. 1022–1031. [Google Scholar]
- Isernia, T.; Leone, G.; Pierri, R. Radiation pattern evaluation from near-field intensities on planes. IEEE Trans. Antennas Propag.
**1996**, 44, 701. [Google Scholar] [CrossRef] - Zhang, H.; Liang, Y. Reshaped Wirtinger flow for solving quadratic systems of equations. Adv. Neural Inf. Process. Syst.
**2016**, 29, 2622–2630. [Google Scholar] - Wang, G.; Giannakis, G.B.; Eldar, Y.C. Solving systems of random quadratic equations via truncated amplitude flow. IEEE Trans. Inf. Theory
**2018**, 64, 773–794. [Google Scholar] [CrossRef] - Wang, G.; Giannakis, G.B.; Saad, Y.; Chen, J. Phase retrieval via reweighted amplitude flow. IEEE Trans. Signal Process.
**2018**, 66, 2818–2833. [Google Scholar] [CrossRef] - Wang, G.; Giannakis, G.B. Solving random systems of quadratic equations via truncated generalized gradient flow. In Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016; pp. 568–576. [Google Scholar]
- Yeh, L.H.; Dong, J.; Zhong, J.; Tian, L.; Chen, M.; Tang, G.; Soltanolkotabi, M.; Waller, L. Experimental robustness of Fourier ptychography phase retrieval algorithms. Opt. Express
**2015**, 23, 33214–33240. [Google Scholar] [CrossRef] - Grohs, P.; Koppensteiner, S.; Rathmair, M. Phase retrieval: Uniqueness and stability. SIAM Rev.
**2020**, 62, 301–350. [Google Scholar] [CrossRef] - Kocsis, P.; Shevkunov, I.; Katkovnik, V.; Egiazarian, K. Single exposure lensless subpixel phase imaging: Optical system design, modelling, and experimental study. Opt. Express
**2020**, 28, 4625–4637. [Google Scholar] [CrossRef] - Shevkunov, I.; Katkovnik, V.; Petrov, N.V.; Egiazarian, K. Super-resolution microscopy for biological specimens: Lensless phase retrieval in noisy conditions. Biomed. Opt. Express
**2018**, 9, 5511–5523. [Google Scholar] [CrossRef] - Kocsis, P.; Shevkunov, I.; Katkovnik, V.; Rekola, H.; Egiazarian, K. Single-shot pixel super-resolution phase imaging by wavefront separation approach. Opt. Express
**2021**, 29, 43662–43678. [Google Scholar] [CrossRef] - Chang, X.; Bian, L.; Gao, Y.; Cao, L.; Suo, J.; Zhang, J. Plug-and-play pixel super-resolution phase retrieval for digital holography. Opt. Lett.
**2022**, 47, 2658–2661. [Google Scholar] [CrossRef] - Chang, X.; Jiang, S.; Zheng, G.; Bian, L. Deep distributed optimization for blind diffuser-modulation ptychography. Opt. Lett.
**2022**, 47, 3015–3018. [Google Scholar] [CrossRef] - Parikh, N.; Boyd, S. Proximal algorithms. Found. Trends Optim.
**2014**, 1, 127–239. [Google Scholar] [CrossRef] - Gao, Y.; Cao, L. A Complex Constrained Total Variation Image Denoising Algorithm with Application to Phase Retrieval. arXiv
**2021**, arXiv:2109.05496. [Google Scholar] - Nesterov, Y.E. A method for solving the convex programming problem with convergence rate O (1/k 2). Dokl. Akad. Nauk SSSR
**1983**, 269, 543–547. [Google Scholar] - Beck, A.; Teboulle, M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci.
**2009**, 2, 183–202. [Google Scholar] [CrossRef] [Green Version] - Xu, R.; Soltanolkotabi, M.; Haldar, J.P.; Unglaub, W.; Zusman, J.; Levi, A.F.; Leahy, R.M. Accelerated Wirtinger flow: A fast algorithm for ptychography. arXiv
**2018**, arXiv:1806.05546. [Google Scholar] - Bostan, E.; Soltanolkotabi, M.; Ren, D.; Waller, L. Accelerated Wirtinger flow for multiplexed Fourier ptychographic microscopy. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 3823–3827. [Google Scholar]
- Zhou, Y.; Zhang, H.; Liang, Y. Geometrical properties and accelerated gradient solvers of non-convex phase retrieval. In Proceedings of the 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA, 27–30 September 2016; pp. 331–335. [Google Scholar]
- Fabian, Z.; Haldar, J.; Leahy, R.; Soltanolkotabi, M. 3D phase retrieval at nano-scale via accelerated Wirtinger flow. In Proceedings of the 2020 28th European Signal Processing Conference (EUSIPCO), Amsterdam, The Netherlands, 18–21 January 2021; pp. 2080–2084. [Google Scholar]
- Available online: https://holoeye.com/gaea-4k-phase-only-spatial-light-modulator/ (accessed on 20 May 2022).
- Kreutz-Delgado, K. The complex gradient operator and the CR-Calculus. arXiv
**2009**, arXiv:0906.4835. [Google Scholar] - Available online: https://github.com/THUHoloLab/pixel-super-resolution-phase-retrieval (accessed on 20 May 2022).
- Li, R.; Cao, L. Progress in phase calibration for liquid crystal spatial light modulators. Appl. Sci.
**2019**, 9, 2012. [Google Scholar] [CrossRef] [Green Version] - Gao, Y.; Li, R.; Cao, L. Self-referenced multiple-beam interferometric method for robust phase calibration of spatial light modulator. Opt. Express
**2019**, 27, 34463–34471. [Google Scholar] [CrossRef] - Li, R.; Gao, Y.; Cao, L. In situ calibration for a phase-only spatial light modulator based on digital holography. Opt. Eng.
**2020**, 59, 053101. [Google Scholar] [CrossRef] - Horn, R.A.; Johnson, C.R. Matrix Analysis; Cambridge University Press: Cambridge, UK, 2012. [Google Scholar]
- Beck, A. First-Order Methods in Optimization; SIAM: Philadelphia, PA, USA, 2017. [Google Scholar]

**Figure 1.**Forward model of a lensless on-chip microscope. (

**a**) Typical optical configurations that can transfer the phase and subpixel information into the intensity variations at the sensor plane. (

**b**) Diffraction model of the imaging system. Diffraction is calculated via the angular spectrum method, where the diffraction angle $\theta $ and the corresponding Fresnel kernel size are determined by the sampling frequency. (

**c**) Sampling model of the sensor pixels.

**Figure 3.**Lensless on-chip microscope based on phase modulation diversity, which we consider as an example in this work. BS is a beam splitter. ${f}_{1}$ and ${f}_{2}$ denote the focal lengths of Lens 1 and Lens 2, respectively. z denotes the sample-to-sensor distance.

**Figure 4.**Simulation results. (

**a**) Evaluation of the quality improvements by TV regularization. (

**b**) Convergence curves of the AWF algorithm and the non-accelerated Wirtinger flow (WF) algorithm using $K=8$ diversity images.

**Figure 5.**Experimental results. (

**a**) Phase reconstruction of a quantitative phase target. The cross-sectional profiles are indicated by the triangular marks. The red doted lines indicate the ground truth phase induced by the structures. The scale bar is 200 $\mathsf{\mu}$m. (

**b**) Convergence curves of the algorithms using $K=8$ diversity images.

**Figure 6.**Experimental validation on biological samples.(

**a**) Amplitude reconstruction of a section of the uterus of parascaris equorum. The inset shows an image of the stained tissue slide. (

**b**–

**e**) are the enlarged images of (

**a**), corresponding to the prophase, metaphase, anaphase, and telophase of the mitosis, respectively. The upper and lower rows show the non-PSR and PSR reconstruction, respectively. The scale bar is 200 $\mathsf{\mu}$m.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Gao, Y.; Yang, F.; Cao, L.
Pixel Super-Resolution Phase Retrieval for Lensless On-Chip Microscopy via Accelerated Wirtinger Flow. *Cells* **2022**, *11*, 1999.
https://doi.org/10.3390/cells11131999

**AMA Style**

Gao Y, Yang F, Cao L.
Pixel Super-Resolution Phase Retrieval for Lensless On-Chip Microscopy via Accelerated Wirtinger Flow. *Cells*. 2022; 11(13):1999.
https://doi.org/10.3390/cells11131999

**Chicago/Turabian Style**

Gao, Yunhui, Feng Yang, and Liangcai Cao.
2022. "Pixel Super-Resolution Phase Retrieval for Lensless On-Chip Microscopy via Accelerated Wirtinger Flow" *Cells* 11, no. 13: 1999.
https://doi.org/10.3390/cells11131999