Next Article in Journal
Convolutional Neural Network for Segmenting Micro-X-ray Computed Tomography Images of Wood Cellular Structures
Previous Article in Journal
Study of Temperature Effect on Cascade Characteristics of Nanofluidic Energy Absorption System
Previous Article in Special Issue
Hybrid Dark Channel Prior for Image Dehazing Based on Transmittance Estimation by Variant Genetic Algorithm
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Improved Retinex-Theory-Based Low-Light Image Enhancement Algorithm

1
Dalian Naval Academy, Dalian 116013, China
2
College of Information Science and Engineering, Northeastern University, Shenyang 110819, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(14), 8148; https://doi.org/10.3390/app13148148
Submission received: 15 May 2023 / Revised: 5 July 2023 / Accepted: 6 July 2023 / Published: 13 July 2023
(This article belongs to the Special Issue Advances in Digital Image Processing)

Abstract

:
Researchers working on image processing have had a hard time handling low-light images due to their low contrast, noise, and brightness. This paper presents an improved method that uses the Retinex theory to enhance low-light images, with a network model mainly composed of a Decom-Net and an Enhance-Net. Residual connectivity is fully utilized in both the Decom-Net and Enhance-Net to reduce the possible loss of image details. Additionally, Enhance-Net introduces a positional pixel attention mechanism that directly incorporates the global information of the image. Specifically, Decom-Net serves to decompose the low-light image into illumination and reflection maps, and Enhance-Net serves to increase the brightness of the illumination map. Finally, via adaptive image fusion, the reflectance map and the enhanced illuminance map are fused to obtain the final enhanced image. Experiments show better results in terms of both subjective visual aspects and objective evaluation indicators. Compared to RetinexNet, the proposed method shows improvements in the full-reference evaluation metrics, including a 4.6% improvement in PSNR, a 1.8% improvement in SSIM, and a 10.8% improvement in LPIPS. Additionally, it achieved an average improvement of 17.3% in the no-reference evaluation metric NIQE.

1. Introduction

Low-light image enhancement is an important branch of image processing, which is mainly applied in fields such as photo surveillance, intelligent driving, and military applications [1]. The main causes of low-illumination images are low light at night and backlighting. Images produced in such environments are generally characterized by low contrast and high noise, which seriously affect the amount of information that can be reflected in the image. In certain circumstances, low-light images can even become distorted, and black can completely obscure the original image information [2,3,4]. Therefore, restoring the brightness of low-light images and improving image contrast is a meaningful research topic.
Various algorithms have been created for processing low-light images. There are three main types of traditional methods for enhancing images at low light: image defogging, the histogram-based method, and the Retinex-based method. These methods are not very effective in processing images, highlighting problems such as color distortion and noise amplification. Therefore, it is of utmost importance to constrain image noise and equalize image color while enhancing image brightness and contrast, which is a hot topic of research in the field of low-light image enhancement (Figure 1).
The field of image processing has been greatly impacted by deep learning. Research is needed to develop it. It has been observed that certain methods that combine the techniques of deep learning and the Retinex framework for low-light image enhancement have been proven to be effective, such as RetinexNet [5] and R2RNet [6]. However, existing methods suffer from problems such as amplified noise, color distortion, and loss of image details after enhancement. Therefore, we propose a new method for extracting images from the network using an attention mechanism that is different from deep learning and the Retinex theory. Furthermore, we introduce a color consistency loss function to the other loss functions to further improve the image quality. The proposed method’s flowchart can be seen in Figure 2. The main contributions of this paper are as follows:
(1)
We designed a new decomposition module with the introduction of residual connections, aiming to reduce the loss of detailed information during the decomposition process and obtain more accurate illumination and reflection maps.
(2)
An effective method is proposed for low-illumination image enhancement.
(3)
By introducing an attention mechanism to improve the network structure and adding a color consistency loss function, the illumination and reflection images can be effectively obtained via the processing.
(4)
The proposed method is significantly better than the current techniques for improving low-light images when it comes to visual perception and evaluation indexes.
The paper begins with Section 2, which covers the various works related to the Retinex theory. In Section 3, we introduce the proposed model, as well as the loss functions that are utilized. In Section 4, we conduct experimental evaluations, including subjective assessments. The paper concludes in Section 5.
Figure 2. A flowchart of the proposed method.
Figure 2. A flowchart of the proposed method.
Applsci 13 08148 g002

2. Related Work

Numerous techniques have been proposed for enhancing low-light images. Algorithms based on histogram equalization have derived dynamic histogram equalization (HE), adaptive histogram equalization (AHE) [7], and contrast-limited adaptive histogram equalization (CLAHE) [8], among others. HE is a process that increases the contrast of an image by distributing its gray range more evenly. AHE divides the image into several small blocks for histogram equalization processing, which can better improve the local contrast of the image and achieve better enhancement results. CLAHE sets a threshold and only averages the parts exceeding the threshold to each gray level, eliminating the blocky effect that may be caused by AHE. Traditional Retinex-based methods mainly include the single-scale Retinex method (SSR), multi-scale Retinex method (MSR) [9], and MSRCP [10], among others. SSR is carried out by using filters, which are designed to filter the various channels of an image. They are then used to create the illumination image. The low-light image and the illumination image are logarithmically processed and subtracted to achieve the enhancement effect. MSR is equivalent to stacking multiple SSRs, which can be regarded as the weighted sum of multiple SSRs. MSRCP adds color balance, normalization, etc., to the multi-scale MSR results. Low-light image enhancement based on dehazing applies the dehazing algorithm to the reflection image of the low-light image, and after processing the image, it is inverted to obtain the final enhancement effect.
LLNet [11] presents a method that trains a self-encoder using a synthetic image dataset. It achieves noise reduction while still enhancing the image. Fu et al. [12] proposed a network that aims to enhance low-light images using an unsupervised model known as LE-GAN. The network uses a light-aware attention module to reduce the noise and extract various features. They also introduced a loss function that can be used to address the issue of overexposure. Ma et al. [13] proposed a self-calibrating lighting framework that features a lighting learning process with crowd-share for enhancement purposes. In contrast, the proposed self-calibrating lighting framework can significantly reduce computational costs and achieve fast processing. Wei et al. proposed RetinexNet, which combines the Retinex theory with convolutional neural networks to enhance low-light images by enhancing the illumination image and reducing noise in the reflectance image using BM3D. R2RNet [6] is also based on Retinex and deep learning networks, and it designs a residual module (RM) and adds it to its Decom-Net, noise reduction network, and augmentation network to obtain better component results. Some algorithms combine Retinex with zero-shot networks, such as RRDNet [14]. RRDNet is modeled on the theory of Retinex and features the method by decomposing the low-illumination image into illumination, reflection, and noise images, thus achieving noise reduction. The method converts the Decom-Net into a generative network, thus reducing the coupling between the illuminance and reflection images.
The key to the Retinex method is to obtain higher quality illumination and reflection images. The residual network has shown remarkable performance in various image processing tasks due to its unique jump connection structure, which effectively prevents the gradient from exploding or disappearing during the training of convolutional neural networks. We build on this foundation by integrating the residual network into the decomposition network to obtain better decomposition results. Furthermore, we introduce an improved color invariant loss function for enhancing the output. Figure 3 demonstrates the decomposition and enhancement results of the shimmering images.

3. Proposed Method

This paper proposes a method that fully considers the image decomposition and the image quality degradation brought about by the enhancement process. The model diagram of this method is shown in Figure 4, which is divided into a Decom-Net and an Enhance-Net. The low-illumination image is used as the input of the Decom-Net, and the output of the Decom-Net is the illumination image and the reflection image. The decomposed illuminance and reflection images are fed into the Enhance-Net to obtain the enhanced illuminance image. Finally, the decomposed reflection image and the enhanced illuminance image are fused to obtain the final enhancement result.

3.1. Decom-Net

The key to the Retinex theory lies in obtaining high-quality illumination and reflection images, which in turn greatly affect the final enhancement results. Therefore, designing a high-quality decomposition network is one of the main focuses of this paper. The design of residual networks aims to solve the problem of gradient explosion or vanishing that may occur when the depth of convolutional models increases. Stacked layers, which include pooling and convolutional techniques, can result in a loss of detail during the processing of images. The proposed network avoids this issue by having a skip-connection structure. This can help minimize the effects of neural networks on the image quality. The decomposition network proposed in this paper is illustrated in Figure 5, where S denotes the input image, R denotes the reflection image, and I denotes the illumination image. A 1 × 1 convolutional layer is added as a connecting layer in the decomposition network. During the training process, with the guidance of sharing the reflection component, the illumination and reflection images learn to be decomposed from the paired image dataset containing low-light images and truth images.

3.2. Enhance-Net

The Enhance-Net architecture proposed in this paper is illustrated in Figure 6. In order to adjust the local brightness of the illumination map while maintaining consistency in the global illumination, we have utilized multi-scale connection methods extensively. Furthermore, we have introduced the positional pixel attention mechanism, which weighs the positional information of the pixels in the image at a pixel level. Unlike the pixel-level attention mechanism, the positional pixel attention mechanism places greater emphasis on the attention weight of each position. This enhances the robustness and generalization ability of the model while also smoothing out some noise and having a certain denoising effect. It is worth noting that the network also achieved high results in image separation, target classification, and other directions.

3.3. Loss Function

To further improve the decomposition and enhancement effects, this paper adopts a series of loss functions. We will describe each of these loss functions in detail.
The decomposition loss function L D e c o m consists of three components: the reconstruction loss, the reflectance component consistency loss, and the illumination smoothness loss. The formula for the decomposition loss function is as follows:
L D e c o m = L r e + λ 1 L r + λ 2 L i
where L r e denotes reconstruction loss, L r denotes reflection component consistency loss, and L i denotes illumination smoothness loss. λ 1 and λ 2 denote the coefficients of reflection component loss and illumination component loss, respectively.
Based on the decomposition of the low-light image into illumination and reflectance maps, as well as the illumination and reflectance maps of a normal-light image, this paper proposes the following formula for image reconstruction:
L r e = λ 3 i = 1 N R n I l S l 1 + λ 4 i = 1 N R l I n S n 1 + i = 1 N R l I l S l 1 + i = 1 N R n I n S n 1
where λ 3 and λ 4 represent their respective coefficients. S n , S l , R n , I n , R l , and I l represent the normal-light image, low-light image, reflection and illumination maps of the normal-light image, and reflection and illumination maps of the low-light image, respectively.
The reflection component consistency loss is defined as follows:
L r = R l R n 1
The illumination smoothness loss is a minimization model that weights the gradient of the reflectance map, and its expression is as follows:
L i = i I i exp λ g R i 1
where ∇ denotes the horizontal and vertical gradients of the image, and λ g is the balance coefficient.
The enhancement loss function consists of the relighting loss and the color constancy loss, which are expressed as follows:
L E n = L l i g h t + λ 5 L c o l
where L E n denotes enhancement loss, L c o l denotes color constancy loss, and λ 5 is the coefficient of noise reduction loss and color constancy loss.
L l i g h t = R l I ^ S n 1
where I ^ is the output of the Enhance-Net.
The color constancy loss is based on the assumption of the grey world color constancy theory, which assumes that the average color of each sensor channel in an image is gray. The goal of the color constancy loss process is to eliminate any biases that may appear in the enhanced image. The equation for the color constancy loss is as follows:
L c o l = p , q Y T p T q 2 , Y R , G , R , B , G , B
where T p denotes the average intensity value of the p channels in the enhanced image method.

4. Experiments

4.1. Experimental Implementation Details

The method adopted is trained and tested on an NVIDIA GeForce RTX2080 graphics card, combined with an Intel i7-8700 processor and 32G of RAM. The software environment employed Pycharm version 2022, equipped with the Anaconda Python 3.7 interpreter and the Pytorch 1.12.1 framework. The initial learning rate was set to 0.001, with λ 1 and λ 2 both set to 0.1 in Equation (1), λ 3 and λ 4 set to 0.01 in Equation (2), λ 5 set to 0.01 in Equation (5), and the remaining weights set to 1.
The training dataset utilized in this study comprises the LOL dataset [5] and a partially synthesized dataset. The total number of images included in the LOL dataset was 500 pairs. Out of these pairs, 485 were used as training sets, and 15 were used as test sets. This is regarded as the most commonly used image enhancement dataset in the research community. For the synthetic version of the dataset, we collected about 1000 images to compensate for the low number of LOL files. The LOL dataset is obtained by capturing images using a camera with varying exposure times and ISO sensitivity. The dataset primarily consists of indoor images with a resolution of 400 × 600. The synthetic dataset is generated by processing the RAISE image dataset using Adobe Lightroom. The images in this dataset are unprocessed high-resolution RAW images captured by four photographers over a period of three years using three cameras at different locations across Europe, capturing various scenes at different times.
We compare quantitatively with traditional methods Retinex, MSRCP, and deep-learning-based methods such as MBLLEN [15], lightenNet [16], RetinexNet [5], EnlightenGAN [17], RRDNet [14], DSLR [18], ExCNet [19], Zero-DCE, Zero-DCE++ [20], and R2RNet. For the testing dataset, we used the paired LOL dataset, VE-LOL-H dataset, and unpaired LIME dataset. We used reference-based evaluation metrics including peak signal-to-noise ratio (PSNR) [21], structural similarity (SSIM) [21], mean squared error (MSE), learned perceptual image patch similarity (LPIPS), and variance inflation factor (VIF) [22], as well as reference-free natural image quality evaluation metrics such as the Natural Image Quality Evaluator [23] to evaluate the deep-learning-based methods.
PSNR is a supervised evaluation method that measures the quality of an image’s reconstruction. According to the evaluation, the closer two images are to one another, the better the reconstruction’s quality.
The structural similarity of an image is measured by using SSIM, which takes into account the differences in structure, contrast, and brightness between different images. Values range from zero to 1, which indicates a better-quality image.
MSE is an image evaluation metric that calculates the average of the squared differences between image pixels and reference image pixels. A smaller MSE value indicates that the image quality is closer to that of the reference image.
LPIPS is a statistical measure of the perceptual similarity of two images. It is commonly used to evaluate the human eye’s perception of images. Compared to other indices, the LPIPS is closely related to the subjective evaluation of humans.
VIF is an index that combines the statistical and natural data collected by the visual system and the human eye to evaluate the quality of images. It shows a correlation between the two values and the images’ perceived visual quality.
NIQE is a non-reference evaluation metric. Both the PSNR and SSIM datasets are paired, but we need non-reference evaluation metrics for non-paired image datasets. Furthermore, it has been found that the PSNR and SSIM metrics do not apply to image texture detail, and NIQE can effectively bridge this gap. The smaller the value of NIQE, the better the image quality.

4.2. Experimental Comparison

We compare the methods in this paper visually and objectively with traditional enhancement methods and the latest enhancement methods. Figure 7, Figure 8 and Figure 9 show the comparison results of the aforementioned methods on the LOL dataset, VE-LOL-H dataset [24], and LIME dataset [25].
Figure 7 illustrates the effect of different methods on the enhancement of images from the LOL dataset. The original input image in low illumination has very low brightness and significant loss of image information. From Figure 7, it is apparent that some of the methods, such as LightenNet, RRDNet, and DSLR, do not show a significant enhancement effect since the enhanced image is not fully exposed, and the enhancement effect could be better. The color of the images from Retinex, RetinexNet, MBLLEN, and EnlightenGAN is distorted, especially in the case of EnlightenGAN in which the color of the leaves turns yellow, and the color deviation is significant. In contrast, the proposed method was able to enhance the image quality significantly. It also preserved the details of the image.
Figure 8 illustrates the enhancement of indoor images from the VE-LOL-H dataset by different methods. The brightness enhancement of the two conventional methods, Retinex and MSRCP, is more pronounced in Figure 8; however, image noise is amplified, and severe color distortion is present. The brightness enhancement effect of Zero-DCE and Zero-DCE++ is better than the method proposed in this paper, but the overall image is obviously white, and the image brightness is too saturated.
Figure 9 illustrates the results of enhancing images from the LIME dataset, wherein an image with a backlighting effect was selected. The original image exhibits rich color information, while some methods introduced severe distortion in the backlight region. For instance, the Retinex effect resulted in the complete disappearance of white clouds. Furthermore, the white clouds in the background of the DSLR enhancement were considerably attenuated, RetinexNet and MSRCP resulted in a noticeable amplification of noise, and the enhanced images of Zero-DCE and Zero-DCE++ exhibited excessive whiteness and severe color deviation. In contrast to the aforementioned methods, the proposed approach achieved a more balanced enhancement between color brightness and contrast.
This study compares four image datasets, including the LOL [5], LIME [25], VV, and DICM [26] datasets. To ensure a fair comparison, we selected publicly available codes without any modifications, and training datasets were also sourced from the LOL dataset. For methods that use unpaired images for training, such as EnlightenGAN and Zero-DCE, we compared them using pre-trained models provided by their respective authors. We evaluated the LOL dataset’s test set using objective evaluation metrics, such as PSNR, SSIM, MSE, and LPIPS. The specific results are shown in Table 1. The LIME, VV, and DICM datasets only contain low-light images. We evaluated the image quality of these datasets using NIQE, and the specific results are shown in Table 2.
Table 1 shows the comparison results of the PSNR, SSIM, MSE, and LPIPS metrics for the LOL dataset’s test set. From this, we can see that all brightness enhancement methods improved the performance metrics. The PSNR, SSIM, and MSE metrics of this method all scored high, with the PSNR metric leading the second place by 0.029 dB, the SSIM metric leading the second place by 0.002, and the MSE metric leading the second place by 39.287. The LPIPS and VIF metrics were also ranked in third and second place, respectively. This indicates that the enhancement effect of this method is closer to the real image than other methods.
To evaluate the LIME, VV, and DICM datasets without reference evaluation indicators, we used the NIQE indicator as shown in Table 2. Our method ranks in the top three positions for both the LIME and DICM datasets. For a fairer comparison, we calculated the average scores of the three datasets and found that our proposed method outperforms most existing methods.
The results of the paper’s experimental procedures exhibited good scores in both the objective indicator and subjective visual evaluations. The proposed method’s superiority was also highlighted. However, we also need to point out that there is still a slight color distortion phenomenon in comparison with some control images, although it is not very obvious, and this is still a direction for the further development of this work.
In Table 3, we calculated the computational complexity based on the average runtime in the LOL test set. It can be observed that the runtime varies significantly among different methods, with EnlightenGAN being the fastest and RRDNet being slower. Compared to other methods, the proposed approach also outperforms the majority of them. However, achieving real-time enhancement for videos remains a key focus for future research in this field.

4.3. Ablation Study

In order to demonstrate the accuracy of the loss function in our parameter settings, we conducted ablation experiments by experimenting with the values of λ 1 , λ 2 , λ 3 , and λ 4 to investigate their impact on enhancing the images, as shown in Figure 10. From the results, we found that there were significant differences between the compared images, and some of the images had significant color deviation, such as the green book part of some images showing a yellowish hue. To better select experimental parameters, we evaluated the comparison results of various parameters using metrics, as shown in Table 3.
According to the subjective visual comparison in Figure 10, it can be observed that the enhancement effect is closest to the reference image when λ 1 =   λ 2 = 0.1 and λ 3 =   λ 4 = 0.01. According to Table 3, the best results in terms of the SSIM, MSE, and NIQE metrics are achieved when λ 1 =   λ 2 = 0.1 and λ 3 =   λ 4 = 0.01. The algorithm also performs well in terms of PSNR, ranking among the top three. Based on the subjective and objective evaluations from Figure 10 and Table 4, we choose the parameter settings of λ 1 =   λ 2 = 0.1, λ 3 =   λ 4 = 0.01.

4.4. Experimental Discussion

From the above comparison of the subjective evaluation and objective metrics, we can see that the method presented in this paper is a significant improvement compared to existing methods. This paper’s proposed method could be applied to various practical applications. For example, the method can effectively improve the brightness of medical images (ultrasound images, X-rays, etc.) and provide more accurate information. It can be applied to biological experiments to obtain images with high signal-to-noise ratios from low-expression samples, and it can also be applied to surveys of mines, mining caves, etc.

5. Limitations and Discussion

The method proposed in this paper is not perfect and has some shortcomings. As can be seen in Figure 7, Figure 8 and Figure 9, despite being regarded as a better alternative, the suggested method still produces distorted images. This issue was demonstrated in Figure 10. Figure 10 shows a comparison with different parameters, from which it can be seen that the blue part of the folder area in the image differs from the real image, although the rest of the image is the same color, this is still something that needs to be investigated in the future. Figure 10 shows a comparison with different parameters, from which it can be seen that the blue part of the folder area in the image differs from the real image, although the rest of the image is the same color. This is still something that needs to be investigated in the future. As deep learning evolves, creating better networks to solve existing enhancement problems is a future focus of our research.

6. Conclusions

The paper presents a framework that aims to enhance low-light images using the Retinex theory. The proposed framework is composed of two components: the Decom-Net and the Enhance-Net. The first one receives the low-light image and obtains the necessary reflection and illumination images. These images are then fed into the Enhance-Net to enhance the illumination images. The decomposed reflection images are fused with the corrected illumination images to obtain the final output. The proposed network model incorporates residual connectivity and attention mechanisms to reduce image detail loss. It also includes a positional pixel attention mechanism to ensure that the enhanced network does not lose image detail features in the image illumination. Additionally, a constant color loss is proposed to correct possible color bias in the enhanced image. The results of the experiments revealed that the suggested method performed well in capturing subjective visual effects and evaluating different metrics. Compared with RetinexNet, the proposed method increased the PSNR, SSIM, and LPIPS by 4.6%, 1.8%, and 10.8% in the full-reference evaluation index, respectively, and increased the NIQE without reference by an average of 17.3%. Although the proposed method has been shown to improve the color perception of images, further research is needed to develop it.

Author Contributions

Conceptualization, J.W.; methodology, J.W. and H.W.; software, J.W. and H.W.; validation, Y.S. and J.Y.; investigation, J.W. and Y.S.; writing—original draft preparation, J.W. and H.W.; writing—review and editing, Y.S. and J.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are openly available online: https://blog.csdn.net/u014546828/article/details/109256136 (accessed on 24 November 2020).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Fang, M.; Li, H.; Lei, L. A review on low light video image enhancement algorithms. J. Chang. Univ. Sci. Technol. 2016, 39, 56–64. [Google Scholar]
  2. Li, C.; Guo, C.; Han, L.; Jiang, J.; Cheng, M.-M.; Gu, J.; Loy, C.C. Low-light image and video enhancement using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 44, 9396–9416. [Google Scholar] [CrossRef] [PubMed]
  3. Liu, J.; Xu, D.; Yang, W.; Fan, M.; Huang, H. Benchmarking low-light image enhancement and beyond. Int. J. Comput. Vis. 2021, 129, 1153–1184. [Google Scholar] [CrossRef]
  4. Zheng, S.; Ma, Y.; Pan, J.; Lu, C.; Gupta, G. Low-Light Image and Video Enhancement: A Comprehensive Survey and Beyond. arXiv 2022, arXiv:2212.10772. [Google Scholar]
  5. Wei, C.; Wang, W.; Yang, W.; Liu, J. Deep retinex decomposition for low-light enhancement. arXiv 2018, arXiv:1808.04560. [Google Scholar]
  6. Hai, J.; Xuan, Z.; Yang, R.; Hao, Y.; Zou, F.; Lin, F.; Han, S. R2rnet: Low-light image enhancement via real-low to real-normal network. J. Vis. Commun. Image Represent. 2023, 90, 103712. [Google Scholar] [CrossRef]
  7. Pizer, S.M.; Amburn, E.P.; Austin, J.D.; Cromartie, R.; Geselowitz, A.; Greer, T.; ter Haar Romeny, B.; Zimmerman, J.B.; Zuiderveld, K. Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 1987, 39, 355–368. [Google Scholar] [CrossRef]
  8. Reza, A.M. Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J. VLSI Signal. Process. Syst. Signal. Image Video Technol. 2004, 38, 35–44. [Google Scholar] [CrossRef]
  9. Rahman, Z.-u.; Jobson, D.J.; Woodell, G.A. Multi-scale retinex for color image enhancement. In Proceedings of the 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland, 19 September 1996; pp. 1003–1006. [Google Scholar]
  10. Jobson, D.J.; Rahman, Z.-u.; Woodell, G.A. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 1997, 6, 965–976. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Lore, K.G.; Akintayo, A.; Sarkar, S. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 2017, 61, 650–662. [Google Scholar] [CrossRef] [Green Version]
  12. Fu, Y.; Hong, Y.; Chen, L.; You, S. LE-GAN: Unsupervised low-light image enhancement network using attention module and identity invariant loss. Knowl.-Based Syst. 2022, 240, 108010. [Google Scholar] [CrossRef]
  13. Ma, L.; Ma, T.; Liu, R.; Fan, X.; Luo, Z. Toward fast, flexible, and robust low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 5637–5646. [Google Scholar]
  14. Zhu, A.; Zhang, L.; Shen, Y.; Ma, Y.; Zhao, S.; Zhou, Y. Zero-shot restoration of underexposed images via robust retinex decomposition. In Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK, 6–10 July 2020; pp. 1–6. [Google Scholar]
  15. Lv, F.; Lu, F.; Wu, J.; Lim, C. MBLLEN: Low-Light Image/Video Enhancement Using CNNs. In Proceedings of the BMVC, Newcastle, UK, 3–6 September 2018; p. 4. [Google Scholar]
  16. Li, C.; Guo, J.; Porikli, F.; Pang, Y. LightenNet: A convolutional neural network for weakly illuminated image enhancement. Pattern Recognit. Lett. 2018, 104, 15–22. [Google Scholar] [CrossRef]
  17. Jiang, Y.; Gong, X.; Liu, D.; Cheng, Y.; Fang, C.; Shen, X.; Yang, J.; Zhou, P.; Wang, Z. Enlightengan: Deep light enhancement without paired supervision. IEEE Trans. Image Process. 2021, 30, 2340–2349. [Google Scholar] [CrossRef] [PubMed]
  18. Lim, S.; Kim, W. DSLR: Deep stacked Laplacian restorer for low-light image enhancement. IEEE Trans. Multimed. 2020, 23, 4272–4284. [Google Scholar] [CrossRef]
  19. Zhang, L.; Zhang, L.; Liu, X.; Shen, Y.; Zhang, S.; Zhao, S. Zero-shot restoration of back-lit images using deep internal learning. In Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 21–25 October 2019; pp. 1623–1631. [Google Scholar]
  20. Li, C.; Guo, C.; Loy, C.C. Learning to enhance low-light image via zero-reference deep curve estimation. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 44, 4225–4238. [Google Scholar] [CrossRef] [PubMed]
  21. Hore, A.; Ziou, D. Image quality metrics: PSNR vs. SSIM. In Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey, 23–26 August 2010; pp. 2366–2369. [Google Scholar]
  22. Seshadrinathan, K.; Soundararajan, R.; Bovik, A.C.; Cormack, L.K. Study of subjective and objective quality assessment of video. IEEE Trans. Image Process. 2010, 19, 1427–1441. [Google Scholar] [CrossRef] [PubMed]
  23. Wang, S.; Zheng, J.; Hu, H.-M.; Li, B. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 2013, 22, 3538–3548. [Google Scholar] [CrossRef] [PubMed]
  24. Li, C.; Guo, C.; Han, L.; Jiang, J.; Cheng, M.-M.; Gu, J.; Loy, C.C. Lighting the darkness in the deep learning era. arXiv 2021, arXiv:2104.10729. [Google Scholar]
  25. Guo, X.; Li, Y.; Ling, H. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 2016, 26, 982–993. [Google Scholar] [CrossRef] [PubMed]
  26. Lee, C.; Lee, C.; Kim, C.-S. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans. Image Process. 2013, 22, 5372–5384. [Google Scholar] [CrossRef] [PubMed]
  27. Guo, C.; Li, C.; Guo, J.; Loy, C.C.; Hou, J.; Kwong, S.; Cong, R. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 1780–1789. [Google Scholar]
Figure 1. Enhanced effect of the traditional methods.
Figure 1. Enhanced effect of the traditional methods.
Applsci 13 08148 g001
Figure 3. The decomposition results obtained by our method: (a) the low-light image, (b) the reflectance image, (c) the illumination image, (d) the enhanced illumination image, and (e) the enhanced result.
Figure 3. The decomposition results obtained by our method: (a) the low-light image, (b) the reflectance image, (c) the illumination image, (d) the enhanced illumination image, and (e) the enhanced result.
Applsci 13 08148 g003
Figure 4. Flow chart of the algorithm in this paper.
Figure 4. Flow chart of the algorithm in this paper.
Applsci 13 08148 g004
Figure 5. Structure of the Decom-Net.
Figure 5. Structure of the Decom-Net.
Applsci 13 08148 g005
Figure 6. Structure of the Enhance-Net.
Figure 6. Structure of the Enhance-Net.
Applsci 13 08148 g006
Figure 7. The results of a low-light image enhancement procedure on the LOL dataset were obtained. (a) Low-illumination image; (b) Retinex; (c) MSRCP; (d) MBLLEN; (e) LightenNet; (f) RetinexNet; (g) EnlightenGAN; (h) RRDNet; (i) R2RNet; (j) DSLR; (k) ExCNet; (l) Zero-DCE; (m) Zero-ECD++; (n) our method; (o) Truth value.
Figure 7. The results of a low-light image enhancement procedure on the LOL dataset were obtained. (a) Low-illumination image; (b) Retinex; (c) MSRCP; (d) MBLLEN; (e) LightenNet; (f) RetinexNet; (g) EnlightenGAN; (h) RRDNet; (i) R2RNet; (j) DSLR; (k) ExCNet; (l) Zero-DCE; (m) Zero-ECD++; (n) our method; (o) Truth value.
Applsci 13 08148 g007
Figure 8. The results of a low-light image enhancement procedure on the VE-LOL-H dataset were obtained. (a) Low-illumination image; (b) Retinex; (c) MSRCP; (d) MBLLEN; (e) LightenNet; (f) RetinexNet; (g) EnlightenGAN; (h) RRDNet; (i) R2RNet; (j) DSLR; (k) ExCNet; (l) Zero-DCE; (m) Zero-ECD++; (n) our method; (o) Truth value.
Figure 8. The results of a low-light image enhancement procedure on the VE-LOL-H dataset were obtained. (a) Low-illumination image; (b) Retinex; (c) MSRCP; (d) MBLLEN; (e) LightenNet; (f) RetinexNet; (g) EnlightenGAN; (h) RRDNet; (i) R2RNet; (j) DSLR; (k) ExCNet; (l) Zero-DCE; (m) Zero-ECD++; (n) our method; (o) Truth value.
Applsci 13 08148 g008
Figure 9. The results of a low-light image enhancement procedure on the LIME dataset were obtained. (a) Low-illumination image; (b) Retinex; (c) MSRCP; (d) MBLLEN; (e) LightenNet; (f) RetinexNet; (g) EnlightenGAN; (h) RRDNet; (i) R2RNet; (j) DSLR; (k) ExCNet; (l) Zero-DCE; (m) Zero-ECD++; (n) our method.
Figure 9. The results of a low-light image enhancement procedure on the LIME dataset were obtained. (a) Low-illumination image; (b) Retinex; (c) MSRCP; (d) MBLLEN; (e) LightenNet; (f) RetinexNet; (g) EnlightenGAN; (h) RRDNet; (i) R2RNet; (j) DSLR; (k) ExCNet; (l) Zero-DCE; (m) Zero-ECD++; (n) our method.
Applsci 13 08148 g009
Figure 10. Effect of different parameter values on the enhanced images.
Figure 10. Effect of different parameter values on the enhanced images.
Applsci 13 08148 g010
Table 1. Quantitative comparisons were conducted on the LOL dataset using PSNR, SSIM, MSE, and LPIPS as evaluation metrics. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
Table 1. Quantitative comparisons were conducted on the LOL dataset using PSNR, SSIM, MSE, and LPIPS as evaluation metrics. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
MethodPSNRSSIMMSELPIPSVIF
Input7.7730.12112613.6200.5600.266
LightenNet [16]11.9650.6325661.5740.3590.704
MBLLEN [15]17.6550.9301515.9110.2870.956
RetinexNet [5]17.3990.9161565.7460.3630.963
EnlightenGAN [17]17.5390.9191960.0180.3260.938
DSLR [18]14.9350.8383679.3890.4030.862
ExCNet [19]16.3900.9082159.0390.3270.935
RRDNet [14]10.8780.5116966.9920.3360.642
Zero-DCE [27]14.8610.8273281.7160.3350.868
Zero-DCE++ [20]14.6820.8553207.4550.3400.882
R2RNet [6]18.1790.8921498.9520.2720.950
Our method18.2080.9321459.6550.3180.959
Table 2. Comparison of the LIME, VV, and DICM datasets was conducted in terms of NIQE. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
Table 2. Comparison of the LIME, VV, and DICM datasets was conducted in terms of NIQE. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
MethodLIMEVVDICMAvg.
LightenNet [16]4.6813.7293.7354.048
MBLLEN [15]4.8184.2943.4424.185
RetinexNet [5]4.3613.4224.2093.997
EnlightenGAN [17]3.6972.9813.5703.416
DSLR [18]4.0333.6493.3893.690
ExCNet [19]4.1623.7833.0393.661
RRDNet [14]4.5223.8453.9924.120
Zero-DCE [27]3.9283.2173.7163.620
Zero-DCE++ [20]3.8433.3412.8353.339
R2RNet [6]3.7043.0933.5033.433
Our method3.5473.3643.3123.408
Table 3. The average running time of the proposed method for enhancing images in the LOL dataset. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
Table 3. The average running time of the proposed method for enhancing images in the LOL dataset. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
MethodRunning Time (s)
LightenNet [16]0.983
MBLLEN [15]3.975
RetinexNet [5]0.469
EnlightenGAN [17]0.012
DSLR [18]0.942
ExCNet [19]12.748
RRDNet [14]78.386
Zero-DCE [27]0.671
Zero-DCE++ [20]0.747
R2RNet [6]3.704
Our method0.633
Table 4. The results of the evaluation were compared with the different parameters using the NIQE, MSE, PNSR, and SSIM metrics. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
Table 4. The results of the evaluation were compared with the different parameters using the NIQE, MSE, PNSR, and SSIM metrics. The best results are highlighted in bold and underlined. The second-best results are in italics, and the third-best results are in bold.
MethodPSNRSSIMMSENIQE
λ 1 =   λ 2 = 0.01 ,   λ 3 =   λ 4 = 0.0117.4660.9311660.8143.404
λ 1 =   λ 2 = 0.01 ,   λ 3 =   λ 4 = 0.117.5900.9281624.5193.560
λ 1 =   λ 2 = 0.01 ,   λ 3 =   λ 4 = 0.517.2810.9271703.4113.282
λ 1 =   λ 2 = 0.1 ,   λ 3 =   λ 4 = 0.0117.4700.9321592.6013.184
λ 1 =   λ 2 = 0.1 ,   λ 3 =   λ 3 = 0.117.0870.9131850.0283.931
λ 1 =   λ 2 = 0.1 ,   λ 3 =   λ 4 = 0.517.2750.9191711.1963.752
λ 1 =   λ 2 = 0.5 ,   λ 3 =   λ 4 = 0.0117.5600.9231611.5883.461
λ 1 =   λ 2 = 0.5 ,   λ 3 =   λ 4 = 0.116.6230.8981919.4513.930
λ 1 =   λ 2 = 0.5 ,   λ 3 =   λ 4 = 0.517.3940.9201720.7433.406
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wang, J.; Wang, H.; Sun, Y.; Yang, J. Improved Retinex-Theory-Based Low-Light Image Enhancement Algorithm. Appl. Sci. 2023, 13, 8148. https://doi.org/10.3390/app13148148

AMA Style

Wang J, Wang H, Sun Y, Yang J. Improved Retinex-Theory-Based Low-Light Image Enhancement Algorithm. Applied Sciences. 2023; 13(14):8148. https://doi.org/10.3390/app13148148

Chicago/Turabian Style

Wang, Jiarui, Hanjia Wang, Yu Sun, and Jie Yang. 2023. "Improved Retinex-Theory-Based Low-Light Image Enhancement Algorithm" Applied Sciences 13, no. 14: 8148. https://doi.org/10.3390/app13148148

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop