Study on a Low-Illumination Enhancement Method for Online Monitoring Images Considering Multiple-Exposure Image Sequence Fusion

Abstract

In order to improve the problem of low image quality caused by insufficient illumination, a low-light image enhancement method with robustness is proposed, which can effectively handle extremely dark images while achieving good results for scenes with insufficient local illumination. First, we expose the images to different degrees to form a multi-exposure image sequence; then, we introduce global-based luminance weights and contrast-based gradient weights to fuse the multi-exposure image sequence; finally, we use a bootstrap filter to suppress the noise that may occur during the image processing. We employ pertinent assessment criteria, such as the Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), the Average Gradient (AG), and the Figure Definition (FD), to assess how well the method enhances. Experimental results show that PSNR (31.32) and SSIM (0.74) are the highest in pretty dark scenes compared to most conventional algorithms such as MF, BIMEF, LECARM, etc. Similarly, in processing uneven illumination such as “moonlit night” images, the AG (10.21) and the FD (14.54) are at maximum. In addition, other evaluation metrics such as Shannon (SH) are optimal in the above scenarios. In addition, we apply the algorithm in this paper to the online monitoring images of electric power equipment, which can improve the image lightness while recovering the detail information. The algorithm has strong robustness in extremely dark images and natural low-light images, and the enhanced images have minimal distortion and best appearance in different low-light scenes.

1. Introduction

Image processing has received a lot of attention from researchers as a result of the advancement of computer vision, and it has been used in a variety of industries, including artificial intelligence [1], smart cities [2], the power industry [3], and the military [4]. Image acquisition, on the other hand, is easily influenced by the environment, particularly in low-light situations such as darkness, overcast days, and obstructed light, when the overall visual impact is gloomy due to the lack of reflected light, and noise and color imbalance can develop [5]. In particular, the monitoring of the power industry is often due to the low-light of the images taken, and the problem of power accidents cannot be dealt with in time, endangering the safety of power equipment and related staff. As a result, the development and application of image enhancement technology has become critical to solving the image quality problem, as it not only provides people with a better visual experience, but also improves the vision system’s reliability and robustness, making it easier to extract and process image information. Scholars at home and abroad have conducted various studies on image enhancement algorithms in order to address the problem of poor image quality in low-light situations.

Retinex: In Retinex-based image enhancement theory, the observed image can be divided into two components, reflection and illumination, and then both are further enhanced before the reflection and illumination are blended to generate the final high-quality image, according to Retinex-based image enhancement theory [6]. Dong et al. [7] inverted low-illumination images to obtain similar fog maps and used a defogging algorithm to improve the image quality. Wang et al. [8] proposed an enhancement algorithm for non-uniform illumination images (NPE), the image is decomposed into reflection and illumination using the bright-pass filter, which determines the detail and naturalness of the image, respectively, under non-uniform illumination; this method can improve image detail while preserving the image’s naturalness, but noise problems will occur. In the method of Fu et al. [9], a new weighted variational model was proposed for better estimating reflectance and illumination a priori, preserving both in greater detail, and in particular suppressing noise to some extent. For the problem of image noise caused by Retinex model-based algorithms after augmentation, Li et al. [10] introduced a robust Retinex model (RRM), and a Lagrangian-based optimization scheme is used to preserve image edges. In summary, Retinex-based algorithms have the advantages of constant color, large-dynamic-range compression, and high color fidelity. However, because most of these algorithms lack the capacity to retain edges, they may generate a halo effect in some sharp edge regions or over-brighten the entire image.

Image fusion: Another low-illumination image enhancement research avenue is the image fusion strategy [11], which is also the topic of this work. The method based on image fusion first acquires images with various lighting at different times using the same sensor, or employs different methods to acquire images with different lighting for the same time and scene. Finally, as much image information as possible is retrieved from the images taken under various lighting conditions and fused into a single high-quality image, resulting in improved image quality and information usage. In our work, the image fusion technique based on a single image is the subject. Le et al. [12] combined the source image and its logarithmic version to get the enhanced image that has sharper details. Yamakawa et al. [13] fused the original image with the Retinex-processed image, which can provide a high-visibility image in both bright and dark areas. According to the illumination–reflection model and multiscale theory, Wang et al. [14] proposed a colored image correction method based on nonlinear functional transformation, then employed principal component analysis (PCA) to combine the original and processed images, in order to improve the self-adaptive capability of image enhancement for low-light images. Celebi et al. [15] generated over- and under-exposed images using a novel adaptive histogram separation scheme, then used a fuzzy-logic-based technique at the fusion stage, which effectively removed ghosting from motion images.

Above all, Fu et al. [16] adjusted the illumination with the sigmoid function and adaptive histogram equalization, and then employed a multi-scale strategy with varied weights in the fusion process to present a trade-off between detail and local. Guo et al. [17] proposed a simple and efficient algorithm, LIME, where the maximum value in the R, G, and B channels is used to estimate the illumination of each pixel, further refining the initial illumination map as the final illumination map; this algorithm is extremely efficient. Through overexposure of non-uniformly illuminated images after enhancement, Wang et al. [18] obtained a priori multi-layer lightness statistics from high-quality images, and incorporated these into the multi-layer enhancement model, which enhanced the contrast and overall lightness of non-uniformly light images while preserving as much naturalness as feasible. Ma et al. [19] decomposed an image patch into three independent components: signal strength, signal structure, and mean intensity, to produce accurate de-noise and de-ghosting effects, each component is processed separately using color and structure information. Then, Ma et al. [20] created MEF-SSIMc, a new MEF image quality model based on MEF-SSIM, and optimized MEF-SSIMc images by iterative search in the space of all images until convergence. The approach has improved quality on both visual perception and MEF-SSIMc, but the calculation time is slow. Ghosh et al. [21] proposed a variational-based Retinex model, and a bright-pass bilateral filtering (BPBF) Fourier approximation was presented to reduce filtering time (by an order) without sacrificing visual quality. In addition, camera response model (CRM)--based fusion algorithms have also attracted the attention of researchers. Ying et al. [22] designed a weight matrix for image fusion using the light estimation technique, and then introduced a camera response model (CRM) to synthesize multi-exposure images to identify the ideal exposure ratio. Ren et al. [23] proposed LECARM, a novel upgraded framework that combines the camera response model (CRM) and the classic Retinex model, which can effectively reduce color distortion and ultimately result in high quality images.

Deep learning: In addition, low-light image processing based on deep learning is attracting more and more attention. Wei et al. [24] collected a low-light dataset (LOL) containing low-/normal-light images, and proposed a deep reading Retinex-net trained on this dataset, which not only has good visual effects in terms of low light enhancement, but can also well represent image decomposition. Xu et al. [25] built a frequency-based low-light-level image decomposition and enhancement model to recover the low-frequency layer image content while suppressing noise. Atoum et al. [26] proposed a color-based LLIE attention network (CWAN) that selects the key color points in the dark image to be enhanced according to the color frequency in the image. Guo et al. [25] proposed a new zero-reference depth curve estimation method (Zero-DCE), and used it to train lightweight neural network DCE-net to adjust the dynamic range of images. Kwon et al. [27] proposed a new low-light image enhancement method, dark region-aware low-light image enhancement (DALE), which can accurately identify dark areas and enhance their lightness.

In short, when dealing with scenarios with uneven lighting, both Retinex-based and image-fusion-based algorithms produce good processing results. However, if we alter the scene to an extremely dark night, the previously mentioned method will exhibit several issues, including general underexposure, vignetting effect, color shift, and loss of detail, limiting the technique’s practical application. As a result, we propose a robust low-light image enhancement algorithm based on multi-exposure image fusion, aiming to extend the application of low-light image enhancement in different lighting scenarios. First, the input image is processed by logarithmic function and non-complex exponential function, and the two processed images are fused by principal component analysis (PCA) to ensure that the lightness of the bright areas is not over-enhanced while illuminating the dark areas, so that the details of the potential image can be properly revealed. Then, for the problem that the image lightness is still dark, the Sigmoid function is used to increase the overall lightness of the image, and the multi-exposure image is obtained by adjusting the parameters. Moreover, we utilize luminance and gradient weights to combine seven photos of varying exposures to create a high-quality image that considers both luminance and edges. Finally, the fast-guided-filter is applied to the augmented image to suppress any noise that may occur. Experiments demonstrate that our algorithm outperforms the compared algorithms in several metrics and has good robustness for effective application in uneven lighting and extreme darkness. The following three areas are where our work makes the most impact:

• To begin, different from [11,12,13], principal component analysis (PCA) is used to fuse the images processed using logarithmic and non-complex exponential functions, because it can be found that PCA expands the unique features of both images while maintaining the features shared by both images, solving the problem of missing features caused by their direct weighting. A weight based on the image’s overall average lightness and gradients is used to increase the overall lightness while also boosting the local lightness and restoring the details of the source image.
• Secondly, different from [14,15], for the noise problem that appears in the processed image, a fast-guided-filter is used to suppress it and obtain a higher quality output image, because we find that the application of filter can effectively suppress the noise problem in the process of low illumination image processing.
• Different from [16,17], our algorithm shows good robustness for extremely dark images and natural images with low light. It can effectively process extremely dark images and achieve good results in local scenes with insufficient illumination.
• Finally, we apply the algorithm to the power industry monitoring, and get an effective processing result, can effectively improve the overall lightness of the image, so that the restored image has a certain degree of fidelity.

2. Materials and Methods

2.1. Proposed Framework

In this paper, from the perspective of image fusion, multiple exposures are performed on images under shimmering light conditions to obtain image sequences with different light intensities, respectively, and the exposed images are fused using weights based on light intensities as well as gradients, and finally the noise of the images is suppressed using fast guided filter to ultimately obtain high-quality images.

The specific algorithm flow is shown in Figure 1. First, the input image $I$ is logarithmically enhanced and non-complex exponential function enhanced to obtain $I_1$ and $I_2$ respectively; then, $I_1$ and $I_2$ are fused based on PCA (principal component analysis) to obtain the fused image $I_3$; lightness enhancement and normalization are performed on $I_3$ to obtain the image sequence $I_e\left(n\right)$ with different lightness; then, the image sequence $I_e\left(n\right)$ is fused according to the weights $W_l$ and $W_g$ to obtain the fused image $I_f$; finally, the noise in the image is suppressed using the fast bootstrap filter, and finally the high-quality image $I_o$ is obtained.

2.2. Image Multi-Exposure

Most of the existing low-light image imaging models follow the Retinex theory [6], Retinex theory decomposes a given image $S(x,y)$ into a luminance image $L(x,y)$ and a reflection image $R(x,y)$, and follows the following equation:

$$S(x,y)=L(x,y)\times R(x,y)$$

(1)

Based on the above model of Retinex theory, low-illumination images are mostly characterized by uneven illumination and overall darkness. Of these, uneven illumination manifests in the varied light intensities of pixels in different regions, with the potential presence of low-, medium- and high-intensity pixels. To do this, the input image is first enhanced with a logarithmic function that enhances low- and medium-intensity pixels while avoiding over-enhancement of high-intensity pixels [23]:

$$I_1=\text{{}\max (I)+1\text{}}\times \log (I+1)$$

(2)

where $I$ is the input low-illumination image and $I_1$ is the logarithmically enhanced image. In addition, the input image $I$ is processed using a non-complex exponential function [28] that modifies the local contrast while weakening high-intensity pixels in order to optimize the $I_1$ local contrast overpowering problem in the subsequent fusion process, as shown below:

$$I_2=1-\exp (I_1)$$

(3)

where $I_2$ is the image after non-complex exponential enhancement. Since $I_1$ and $I_2$ have similar overall information, in our study, a principal component analysis (PCA)-based method is chosen for image fusion in order to expand their unique information while preserving their overall similarities in information [29]. By computing the feature vectors of the source images and the related feature values, the principal components of similar images are discovered, and the weights of the subimages to be fused are determined based on the principal components [14]. The process is shown in Figure 2.

For two source images, $I_1$ and $I_2$, each image is regarded as an n-dimensional vector denoted as ${\mathit{X}}_m$, where $m=1,2$. The specific PCA image fusion procedure is as follows:

1.

Construct n-dimensional vector matrix X using two source images.

$$\mathit{X}=\left(\begin{array}{ll}{\mathit{X}}_{11}& \hspace{1em}{\mathit{X}}_{21}\\ {\mathit{X}}_{12}& \hspace{1em}{\mathit{X}}_{22}\\ ⋮& \hspace{1em}⋮\\ {\mathit{X}}_{1n}& \hspace{1em}{\mathit{X}}_{2n}\end{array}\right)$$

(4)

2.

Calculate the covariance matrix $\mathit{C}$ of the data matrix $\mathit{X}$.

$$\mathit{C}=\left(\begin{array}{cc}{\mathit{\sigma }}_{11}^2& {\mathit{\sigma }}_{12}^2\\ {\mathit{\sigma }}_{21}^2& {\mathit{\sigma }}_{22}^2\end{array}\right)$$

(5)

In Equation (5), ${\mathit{\sigma }}_{ij}^2$ is the covariance of the image, which satisfies:

$${\mathit{\sigma }}_{ij}^2=\frac{1}{n}{\displaystyle \sum _{k=1}^n(x_{i,k}-\overline{x_i})}(x_{i,\mathrm{k}}-\overline{x_j})$$

(6)

where $\overline{x_i}$ and $\overline{x_j}$ are the average value of the i-th and j-th source image, respectively.

3.

Calculate the eigenvalues ${\lambda }_i$ of the covariance matrix $\mathit{C}$, and the corresponding eigenvectors ${\mathit{\xi }}_i=\left(\begin{array}{l}{\mathit{\xi }}_{i1}\\ {\mathit{\xi }}_{i2}\end{array}\right)$, where $i=1,2$.

4.

Select a large eigenvalue $\lambda =\max ({\lambda }_1,{\lambda }_2)$, and calculate the weight coefficient using the feature vector corresponding to the largest eigenvalue.

$$W_1={\mathit{\xi }}_{i1}/({\mathit{\xi }}_{i1}+{\mathit{\xi }}_{i2}),W_2={\mathit{\xi }}_{i2}/({\mathit{\xi }}_{i1}+{\mathit{\xi }}_{i2})$$

(7)

5.

Based on the above, we fuse the images $I_1$ and $I_2$: $I_3=W_1\times I_1+W_2\times I_2$. The fused image $I_3$ is obtained, which contains the features of images $I_1$ and $I_2$ while retaining the similarity between the two, solving the problem of missing features caused by direct weighting of the two images. However, the lightness of image $I_3$ after PCA fusion is still dark, and the lightness of the image needs to be enhanced.

To enhance the overall lightness of the PCA-fused image $I_3$ in order to display the majority of the image’s potential details, a modified cumulative distribution function of the β hyperbolic secant distribution (BHS) is employed [30].

$$F(x)=\frac{2\arctan \left(\exp \right(x\left)\right)}{\mathsf{\pi }}$$

(8)

The standard BHS function, a very common function in probability theory, can improve the overall lightness as well as enhance the contrast. In addition, to obtain exposure image sequences with varied luminance, a hyperparameter μ is introduced and the original BHS function is modified using the Sigmoid function as follows.

$$I_4=\mathrm{erf}[\mu \times \arctan \left(\exp \right(I_3)-0.5\times I_3\left)\right]$$

(9)

where $I_4$ is the image after I₃’s lightness has been increased. To begin, the curve transformation of the BHS function is increased using the error function erf, which effectively improves the image lightness. Then, instead of 2/π, a hyperparameter μ is used to adjust the degree of lightness amplification, the greater the μ is, the brighter the output image is, and the range of μ is experimentally known as [2,7]. Furthermore, employing $0.5\times I_3$ allows the image tones to be more similar to the tones of the scene as observed by the human eye. However, the image $I_4$ modified by Equation (9) has a huge dynamic range, causing the image to appear very white. To accomplish this, the image $I_4$ must be normalized in order to keep the dynamic range within the usual range.

$$I_e(n)=\frac{I_4-I_4^{\min }}{I_4^{\max }-I_4^{\min }}$$

(10)

where $I_e\left(n\right)$, $n=2,3,5,6,7$ is the normalized image sequence. Compared with $I_4$, $I_e\left(n\right)$ has a more standard dynamic range and visually better results.

2.3. Image Fusion

Although adjusting the value of μ can achieve relatively satisfactory exposure images after using the above method to generate image sequences with varying exposure intensities, there are still issues such as insufficient detail, overexposure, and contrast imbalance, among others. For this reason, it is necessary to fuse the resulting multi-exposure image sequences. Most traditional multiple-exposure image fusion methods define a weighting function for multiple images, assign larger weights to the well-exposed regions, and define smaller weights in the poorly exposed regions to achieve global lightness enhancement by weighting, and the traditional image fusion methods are as follows:

$$I_{fus\mathrm{ed}d}={\displaystyle \sum _{n=1}^N{W}_n\times }{I}_n$$

(11)

where N is the number of images, $I_n$ is the pixel intensity of the nth image, and $W_n$ is the weight of the image $I_n$. For multiple exposure images, in the traditional method, the same function is basically applied to define the weights.

Unlike traditional methods, in this paper, weights are defined separately using two different functions for multi-exposure image fusion. Specifically, weights $W_l$ characterize the relevance of pixel intensity at the measurement point in relation to the overall average luminance and the intensity of nearby pixels, while weights $W_g$ characterize the importance of pixels at smaller gradients. Finally, the multi-exposure images are fused using pyramids to obtain the image $I_f$, the specific method of multi-exposure fusion is as follows.

2.3.1. Weight Design Based on the Average Luminance

The design of luminance weights is significant in multi-exposure image fusion, often providing more weight to pixels in darker regions and vice versa. Distinguishing between the luminance weights of 0 (underexposed) and 1 (overexposed) proposed in refs. [13,31] used a Gaussian function that allocated a weight to each pixel based on how near the lightness was to the reference value μ, as indicated in Equation (12).

$$f(x,y)=\exp (-\frac{{\left[I\right(x,y)-\mu ]}^2}{2{\mathsf{\sigma }}^2})$$

(12)

where $I(x,y)$ is the luminance value of each pixel of the input image, μ is the mean (reference value) of the Gaussian function, and σ is the standard deviation of the Gaussian function; in ref. [13] the authors set values of μ = 0.5 and σ = 0.2. When the input pixel lightness is close to the reference value μ, the weight is increased; conversely, when the input pixel lightness is the principal reference value μ, the weight is decreased. However, in dark or bright regions far from 0.5, Equation (12) cannot assign a bigger weight to the pixel lightness; specifically, the weight in Equation (12) cannot highlight the dark regions of an image that is brighter overall or the bright regions of an image that is dark overall. To solve the above problem, a weight $W_l$ relative to the overall average image lightness $I_{\mathrm{mean}}$ is introduced, and when $I(x,y)$ is close to $(1-I_{\mathrm{mean}})$, a large weight is given to the pixel point.

$$W_l(x,y)=(\frac{1}{\sqrt{2\mathsf{\pi }}\sigma }\left)\exp \right(-\frac{{\left[I\right(x,y)-(1-I_{\mathrm{mean}}\left)\right]}^2}{2{\sigma }^2})$$

(13)

Similarly, in Equation (13), we set the standard deviation σ of the Gaussian function to be 0.5. In order to explain the standard deviation σ, we performed a value experiment.

As can be seen from Figure 3, when σ = 0.1, the lightness of many shadow parts in the image is still too low; when σ = 1, the image has the problem of missing detail due to the increase in lightness. The lightness and detail problems caused by the value of σ are more obvious in the red rectangles.

2.3.2. Weight Design Based on the Global Gradient

In a poorly exposed image, the intensity of dark pixels is close to zero, while bright areas usually exhibit a high contrast (gradient) due to their high pixel intensity. Conversely, the contrast (gradient) in the darker areas is greater in the better-exposed images. The texture detail in the region with greater contrast (gradient) will acquire more weight as a result of the usage of Formula (13); nevertheless, the texture detail in well-exposed regions with little contrast (gradient) may be under-appreciated due to the lesser weight. To improve this problem, gradient weight $W_g$ is used to emphasize well-exposed areas without regard to their local contrast [32].

$$W_g(x,y)=\frac{Gra{d}_n{\left(I_n\right(x,y\left)\right)}^{-1}}{{\displaystyle {\sum }_{n=1}^{N}Gra{d}_n{\left(I_n\right(x,y\left)\right)}^{-1}+\epsilon }}$$

(14)

where ε is a very small positive value to prevent the denominator from becoming zero. $Gra{d}_n{\left(I_n\right(x,y\left)\right)}^{-1}$ is the gradient at pixel intensity I_n(x, y). With this weight $W_g$, the small gradients of the image can be well described and the information loss at the small gradients can be reduced.

2.3.3. Fusion Based on Pyramid

The final weight $W_f$ is calculated by combining the luminance weight $W_l$ and the gradient weight $W_g$.

$$W_f(x,y)=\frac{W_l{(x,y)}^{{\gamma }_1}\times W_g{(x,y)}^{{\gamma }_2}}{{\displaystyle {\sum }_{n=1}^N{W}_l{(x,y)}^{{\gamma }_1}\times W_g{(x,y)}^{{\gamma }_2}+\epsilon }}$$

(15)

where ${\gamma }_1$ and ${\gamma }_2$ are used to measure the importance of the luminance and the gradient; both are set to 1 in this paper. Finally, using the weight $W_f$, the multiple exposure image sequences are combined with Laplace pyramids to produce the output image $I_p$ [33]. The specific Laplace and Gaussian Pyramid fusion method is as follows.

The N weights are first normalized to have a sum of 1 at each pixel, i.e.,

$${\stackrel{⌢}{W}}_{xy,n}=\frac{W_{xy,n}}{{\displaystyle {\sum }_{n \text{′}=1}^N{W}_{xy,n \text{′}}+\epsilon }}$$

(16)

The input image is split into Laplace pyramids, with each layer comprising band-pass-filtered images at various scales, where each layer is then fused separately. The l-th level of the Laplace pyramid in the image sequence $I_e$ is defined as $L{\text{{}{I}_e\text{}}}_{x,y}^L$ and the lth level of the normalized weight ${\stackrel{⌢}{W}}_{xy,n}$ Gaussian pyramid is defined as $G{\text{{}{\stackrel{⌢}{W}}_{xy,n}\text{}}}_{x,y}^L$. From this, the pixel intensity of each layer of the fused image can be defined.

$$L{\text{{}I\text{}}}_{xy}^n={\displaystyle {\sum }_{n=1}^{N}L{\text{{}{I}_e\text{}}}_{xy}^n}\times G{\text{{}\stackrel{⌢}{W}\text{}}}_{xy}^n$$

(17)

The nth layer of the Laplace pyramid is used as the original image pixel intensity, and the nth layer of the normalized-weight Gaussian pyramid is used as the weight. Finally, the Laplacian pyramid $L{\text{{}{I}_e\text{}}}_{x,y}^L$ of the fused image is recursively obtained as a high-quality image $I_p$. For the different band features and details of different layers of the image Laplace pyramid, the corresponding weighted Gaussian pyramid is used, and the features and details are highlighted by fusion to finally obtain a high-quality fused image $I_p$.

Finally, for the problem of noise in the fused image, a fast guided filter [34] is used to suppress the noise while maintaining the edges, yielding the output image $I_o$. In addition, the whole process is summarized in Algorithm 1.

Algorithm 1

Input: input image I

Do    1. Obtain I1 and I2 respectively by logarithmic and non-complex exponential function enhancement using Equations (2) and (3); 2. Obtain I3 by fussing images I1 and I2 based on the PCA; 3. Multiple exposures for I3 to get sequences of images Ie (n) using Equations (9) and (10), where n = 2, 3, 4, 5, 6, 7; 4. Fusing Ie (n) according to Wl and Wg; 5. Suppress the noise using fast-guided-filter.

Output: enhanced image Io

3. Experimental Results and Analysis

3.1. Dataset and Experimental Environment

We used a computer with a Core i7, 3.40 GHz CPU and used the MATLAB 2020 simulation software to process the low-illumination image. The proposed algorithm is compared with the traditional algorithms Dong [7], NPE [8], SRIE [9], RRM [10], MF [16], BPBF [21], BIMEF [22], BIMEF [22], and LECARM [23]. The proposed algorithm was tested on the low-light paired dataset (LOL), which is the first dataset containing image pairs taken from real scenes for low-light enhancement, and contains 500 low-/normal-light image pairs, which were resized to 400 × 600.

3.2. Comparison with Others

Firstly, the qualitative and quantitative analysis of very dark images on the basis of a standard reference is performed using different algorithms. In addition, to verify the robustness of the method, low-illumination images with different light intensities under natural light are selected for processing and compared with the reference-free evaluation index.

3.2.1. Processing Extremely Dark Images

The algorithm proposed in this paper focuses on very dark images. Firstly, a very dark image with a reference is selected for qualitative and quantitative analysis, and six other low-illumination image enhancement algorithms are compared: fusion-based low-light enhancement algorithms, including BIEMF [22] and MF [16]; the algorithm for the simultaneous use of reflection and illumination estimation—SRIE [9]; a camera response model (CRM)-based algorithm—LECARM [23]; an algorithm based on bright-pass bilateral filtering—BPBF [21]; and a fast and efficient low-light video enhancement method—Dong [7]. Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 show the results of the processing of the extremely dark images using different algorithms, including the input images as well as the standard reference images.

In Figure 4, the algorithms of BIMEF [22], LECARM [23], BPBF [21], MF [16], and SRIE [9] all fail to cause the image to be well exposed, while Dong’s algorithm [7] achieves a better exposure, but the processed image looks more like a painting. In contrast, the algorithm presented in this paper processes the image with good exposure, which is reflected especially in the white door that is open.

The same situation appears in Figure 5, where the image is darker overall, especially in the SRIE [9]-processed image (Figure 5h). Additionally, the details of the results of MF [16] (Figure 5g), Dong [7] (Figure 5i) and the algorithm presented in this paper (Figure 5c) are enlarged. As shown in Figure 6, the enlarged image details reflect the noise of the results of MF [16] and Dong [7], and the more satisfactory visual effect of the algorithm presented in this paper results in the removal of noise.

In addition to the same darker lightness and noise as the above figure, in Figure 8, we zoom in on the details of MF [16] results (Figure 8b) and the LECARM [23] result (Figure 8c) to compare them with the results obtained using the algorithm presented in this paper (Figure 8a). The zoomed-in detailed image reflects that the LECARM [23] and MF [16] result images have heavier shadows above the collar and blurring at the edge textures (e.g., green hanger), while this paper’s algorithm can significantly increase the local lightness of the image and keep the edges, and has better visual effects at that level of detail.

In Figure 9, the color distortion problem is illustrated by comparing the reference image (Figure 9b) with the color of the “white toy” after processing using the algorithm in Figure 9.

In addition to the small scenes above, we widened the scene range and processed Figure 10a using different algorithms. Compared to other methods, the algorithm proposed in this paper has better results at improving the overall and local lightness of the image, eliminating noise, maintaining edge details, and preventing excessive color distortion.

Since image evaluation is closely related to human visual perception, it is difficult to find a universal metric for quantifying the quality of enhanced images. In general, image quality assessment (IQA) can be divided into fully referenced and unreferenced. For quantitative evaluation of images with standard reference and real images, we use full reference IQA and no reference IQA in this research. For reference IQA, we employ peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and the patch-based contrast quality index (PCQI) [35] to evaluate the quality of the enhanced image on the basis of the reference image. For the reference-free IQA, we employ average gradient (AG) [36], figure Definition (FD), spatial frequency (SF) [37], edge strength (EI) and Shannon (SH) as evaluation criteria.

For the extremely dark images with references, we selected ten to calculate their PSNR, SSIM, PCQI, AG, EI, FD, SF, and SH, and took the average of the evaluation indexes of these ten images. As shown in Figure 11, it can be seen that the algorithm proposed in this paper has the highest performance in all eight metrics, which indicates that the images enhanced using the algorithm proposed in this paper have the least distortion and the best appearance.

3.2.2. Processing Low-Light Natural Images

In this paper, in order to demonstrate the robustness of the algorithm when processing low-illumination images, we select three natural non-standard low-illumination images and process those three-low illumination images “moon night”, “house” and “clock” using different algorithms—BIMEF [22], MF [16], SRIE [9], LECARM [23]. BPBF [21], NPE [8] and RRM [10]; the processing results are shown in Figure 12, Figure 13, Figure 14 and Figure 15. Subjectively, it can be seen that the algorithm in this paper can result in the image receiving better exposure and the edge details being refined, which is an excellent result. Next, we analyze the images processed using different algorithms using no-reference IQA, thus obtaining

Table 1. Evaluation index of images processed using different algorithms for “moon”.

Method	AG	EI	FD	SF	SH
Ours	10.21	96.51	14.54	32.80	7.78
BIMEF	6.95	65.87	9.92	21.17	7.48
RRM	7.76	73.88	10.98	22.68	7.53
LECARM	9.11	86.60	12.98	28.00	7.51
BPBF	6.97	65.98	9.95	20.31	7.47
MF	8.08	76.08	11.62	24.46	7.58
SRIE	6.07	58.49	8.49	18.89	7.38
NPE	6.57	62.34	9.35	19.19	7.37

,

Table 2. Evaluation index of images processed using different algorithms for “house”.

Method	AG	EI	FD	SF	SH
Ours	10.41	105.52	13.19	28.22	7.78
BIMEF	7.80	80.61	9.64	21.76	7.28
RRM	8.32	88.03	10.85	22.39	7.48
LECARM	10.34	106.32	12.87	28.06	7.43
BPBF	8.32	85.13	10.44	21.14	7.71
MF	9.57	97.82	12.03	25.34	7.82
SRIE	7.82	81.66	9.44	22.36	7.62
NPE	8.57	87.65	10.79	22.53	7.59

and

Table 3. Evaluation index of images processed using different algorithms for “clock”.

Method	AG	EI	FD	SF	SH
Ours	6.99	74.36	8.25	20.79	7.39
BIMEF	4.11	43.47	4.88	11.88	6.48
RRM	6.26	66.48	7.36	17.98	6.55
LECARM	4.89	43.60	4.79	12.68	6.38
BPBF	4.22	44.77	4.97	11.50	6.26
MF	5.49	58.08	6.51	15.67	6.58
SRIE	3.33	35.47	3.89	10.26	6.07
NPE	5.54	58.83	6.54	15.50	6.51

, for a more intuitive comparison of the IQA of each algorithm, the best values are bolded. As can be seen from Figure 12, Figure 13, Figure 14 and Figure 15, the algorithm in this paper can improve the image lightness effectively while preserving the details of the source image. As shown in the tables, when compared to other algorithms, this paper’s algorithm has higher values in the objective evaluation index, indicating that the enhanced image presented in this paper has the best appearance and less distortion, reflecting the algorithm’s strong robustness in processing low-illumination images with different degrees of illumination. In particular, however, the LECARM [23] algorithm achieved higher values for the edge intensity EI evaluation metric in the evaluation metric of the “house” image, as explained in Figure 14. The LECARM [23] algorithm is more detailed when it comes to edge processing than the algorithm presented in this paper, and this is something that needs to be enhanced in future research.

3.3. Ablation Experiment

In order to further verify the importance of introducing two weight functions W_l (luminance weight) and W_g (gradient weight), we select different luminance images for the ablation experiment and assess the importance of the two weights in the fusion process through subjective evaluation as well as objective evaluation. Ablation experiment I carries out image fusion using only the luminance weight W_l, while the ablation experiment II carries out image fusion using only the gradient weight W_g. The experimental results are shown in Figure 16 and Figure 17. In the “house” image with more textures, comparing Figure 16a,b, the blurring of the edge textures (in the red box) due to the loss of gradient weights W_l in Figure 16b can be clearly seen; in addition, comparing Figure 16a,c, in terms of contrast of luminance, the visual effect of weak local contrast (in the green box) due to the lack of luminance function can be observed in Figure 16c.

Since the changes in luminance and local contrast are not clearly shown in Figure 16, we employed Figure 17 to further illustrate the importance of luminance weighting (yellow box). It can be seen that the luminance of the streetlight in Figure 17c was significantly reduced, and is very similar to the color of the surrounding trees, reflecting the problem of low local contrast due to the lack of luminance weights W_l.

In the following, we quantify the importance of luminance weights W_l and gradient weights W_g by objective evaluation indexes. We use nine images and calculate the values of PSNR, SSIM, PCQI, EI, AG, and SH, and take the average value. As shown in

Table 4. Comparison of mean evaluation indexes of images after two ablation experiments.

Experiment	Ours	Ex_I	Ex_II
PSNR	31.09	29.24	29.79
SSIM	0.74	0.74	0.71
PCQI	0.62	0.60	0.59
AG	9.37	5.82	7.67
EI	76.03	56.02	67.29
SH	7.31	7.26	7.11

and Figure 18, Experiment I is better than Experiment II in terms of PCQI and SSIM, which contain indicators for evaluating image signal strength, objectively reflecting the effect of luminance weight W_l on the improvement of overall image lightness and local contrast enhancement. Conversely, Experiment II outperforms Experiment I in evaluating the AG and EI of the edge gradients, objectively reflecting the importance of the gradient weights W_g for edge detail processing.

3.4. Application in Electric Power Equipment

As the monitoring devices on power equipment such as lines and insulators are frequently affected by environmental factors such as rain, fog, darkness, and so on, the monitoring devices cannot alert the staff and equipment in time when the power equipment fails, which is especially common in dark environments. In this regard, we apply the algorithm proposed in this paper to the detection of power equipment such as transmission lines and insulators in dark and uneven lighting scenarios, and the processing results are shown in Figure 19.

As shown in Figure 19, when transmission lines, insulators, substations, and other power equipment are dark, as illustrated in Figure 19a,b, the technique in this study is able to successfully improve image lightness while recovering related detail information. Furthermore, as demonstrated in Figure 19c,d, the algorithm described in this research is also relevant to uneven illumination settings, demonstrating the program’s robustness in coping with dark conditions. The results in Figure 18 show that the method described in this paper is effective at processing electrical equipment maps under surveillance photography in dark conditions and recovering the related detailed information while improving the image lightness.

4. Conclusions

In this paper, we proposed a low-illumination image enhancement method for non-uniformly illuminated, extremely dark scenes. Compared with most Retinex algorithms and image fusion algorithms, the algorithm in this paper had better robustness under different illumination conditions, demonstrating practical engineering implications in computer vision applications. The algorithm in this paper used luminance weights and gradient weights based on the overall image lightness to fuse enhanced images with different exposures to illuminate dark areas while ensuring that the bright areas were not over-enhanced, and that the details of potential images were properly revealed. In addition, the experiments showed that the algorithm proposed in this paper outperformed most conventional low-illumination algorithms, probably because the algorithm in this paper is applicable to a wider range of scenes, and the enhanced image can both enhance the contrast and capture enough image information. Finally, we applied the algorithm to the online monitoring image of power equipment such as lines and insulators under dark conditions, and effectively recovered picture lightness as well as detail information.