Next Article in Journal
End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++
Next Article in Special Issue
Multiscale Spatial-Spectral Convolutional Network with Image-Based Framework for Hyperspectral Imagery Classification
Previous Article in Journal
Mapping Invasive Phragmites australis in the Old Woman Creek Estuary Using UAV Remote Sensing and Machine Learning Classifiers
Previous Article in Special Issue
A Novel Effectively Optimized One-Stage Network for Object Detection in Remote Sensing Imagery
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Adaptive Contrast Enhancement for Infrared Images Based on the Neighborhood Conditional Histogram

School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2019, 11(11), 1381; https://doi.org/10.3390/rs11111381
Submission received: 30 April 2019 / Revised: 1 June 2019 / Accepted: 5 June 2019 / Published: 10 June 2019

Abstract

:
In this paper, an adaptive contrast enhancement method based on the neighborhood conditional histogram is proposed to improve the visual quality of thermal infrared images. Existing block-based local contrast enhancement methods usually suffer from the over-enhancement of smooth regions or the loss of some details. To address these drawbacks, we first introduce a neighborhood conditional histogram to adaptively enhance the contrast and avoid the over-enhancement caused by the original histogram. Then the clip-redistributed histogram of the contrast-limited adaptive histogram equalization (CLAHE) is replaced by the neighborhood conditional histogram. In addition, the local mapping function of each sub-block is updated based on the global mapping function to further eliminate the block artifacts. Lastly, the optimized local contrast enhancement process, which combines both global and local enhanced results is employed to obtain the desired enhanced result. Experiments are conducted to evaluate the performance of the proposed method and the other five methods are introduced as a comparison. Qualitative and quantitative evaluation results demonstrate that the proposed method outperforms the other block-based methods on local contrast enhancement, visual quality improvement, and noise suppression.

1. Introduction

With the ability to convert the passively received infrared radiation into infrared images, infrared imaging systems have been widely used in military and civilian fields, for instance, thermal remote sensing, weapon guidance, night vision, fire detection, and disease diagnosis [1,2,3,4,5]. In these fields, infrared imaging has superiority and application prospects that are incomparable to visual imaging. However, compared with visual images, the raw infrared images usually suffer from poor contrast and low resolution, which are the main factors that limit the performance of infrared imaging systems [6]. Therefore, contrast enhancement is one of the indispensable pre-processing methods in infrared imaging systems to improve the visual quality of infrared images [7].
Contrast enhancement methods for infrared images have been studied for decades, and most of them are based on the most famous and classical method histogram equalization (HE) [8]. According to the scope of the mapping function of the contrast enhancement method, these methods can be classified into two categories: global contrast enhancement (GCE) and local contrast enhancement (LCE) [9]. In the GCE methods, one global mapping function which corresponds to the entire image characteristics is applied to each pixel of the image. On the contrary, in LCE methods, multiple local mapping functions are used to enhance the contrast of their corresponding areas of the image.
Among the GCE methods, HE is the most well-known one and has been widely applied due to its low computational complexity and effectiveness. In HE, the contrast enhancement ratio of each grayscale is proportional to its probability distribution function. Therefore, the enhanced result may appear over-enhancement in the homogeneous regions, especially when the homogeneous regions account for a large proportion in the infrared images [10]. To address this drawback, there are many improved HE methods investigated. The plateau histogram equalization (PHE) [11,12], double plateaus histogram equalization (DPHE) [13], and adaptive double plateaus histogram equalization (ADPHE) [14] attempt to limit the contrast by introducing one or two proper thresholds to avoid over-enhancement. Other methods, such as the brightness preserving bi-histogram equalization (BBHE) [15], dualistic sub-image histogram equalization (DSIHE) [16], recursive mean-separate histogram equalization (RMSHE) [17], adaptive histogram segmentation (AHS) [7], and adaptive histogram partition (AHP) [18], adaptively divide the histogram into two or more intervals, then carry out histogram equalization in each interval within the assigned grayscale range. Additionally, the histogram modification framework (HMF) [19] and histogram specification (HS) [20] are also effective methods to avoid the over-enhancement of HE. The GCE methods generally enhance the contrast based on the entire image characteristics with low computational complexity and favorable light order preservation. However, the global mapping function is not suitable for various regions with different local contrast characteristics, which is the limitation of GCE methods.
To improve the performance of GCE methods, the LCE methods are developed to enhance the local contrast based on the characteristics of local regions. The two representative methods of them are the contrast limited adaptively histogram equalization (CLAHE) [21] and partially overlapped sub-block histogram equalization (POSHE) [22]. In CLAHE, the input image is divided into multiple non-overlapped sub-blocks. In each sub-block, the histogram is clipped and redistributed to avoid over-enhancement. In addition, to avoid the appearance of block artifacts, bilinear interpolation is carried out between mapping functions of the adjacent sub-blocks. In POSHE, the input image is divided into multiple partially overlapped sub-blocks. Histogram equalization is carried out in each sub-block, for the pixels in overlapped regions, the enhanced results are the average value of correlated sub-blocks. As an improvement of CLAHE, the balanced CLAHE and contrast enhancement (BCCE) [23] improves the redistribution mechanism and introduces an exponential factor to enhance the local contrast. Nevertheless, the block artifacts and noise amplification may still appear in the large homogeneous regions of the enhanced results. Therefore, the adjacent-blocks-based modification for local histogram equalization (ABMHE) [24] and local gradient-grayscale statistical feature (LGGSF) [25] take the content of each sub-block into consideration when calculating the corresponding mapping function. In ABMHE, the partially overlapped sub-blocks are categorized into three categories based on the gradient information. Not only the category of each sub-block but also that of its adjacent sub-blocks are taken into consideration when calculating the mapping function. In LGGSF, the non-overlapped sub-blocks are categorized into two categories using the classification model trained by a support vector machine (SVM). The mapping function of each sub-block is calculated according to its category. These classification-based LCE methods get better performance on eliminating the block artifacts. However, the improper classification in the transition regions or the low contrast regions may result in the loss of detail. In addition to the block-based methods, the adaptive trilateral contrast enhancement (ATCE) [26] manipulates the contrast, sharpness, and intensity based on the modified histogram and the extracted feature of the input image. Other models, such as the unsharp masking [27,28], retinex theory [29], and wavelet transform [30,31,32], can also be utilized to improve the visual quality of infrared images. These LCE methods produce better local contrast enhanced result than GCE methods, however, they may still suffer from noise amplification, block artifacts, and/or loss of details.
In our previous paper [33], we tried to improve the visual quality of infrared images by combining the merits of both GCE and LCE methods. In that paper, the global and local histogram specification (GLHS) enhances the contrast of infrared images by applying histogram specification globally and locally based on the 2D histogram of the input image. Then the local contrast is enhanced based on both global and local enhancement results. Whereas the utilization of 2D histogram makes GLHS a time-consuming method, we found the global–local conception can be utilized to improve the CLAHE method. Therefore, in this paper, an adaptive contrast enhancement method based on the neighborhood conditional histogram is proposed to improve the visual quality of infrared images. First, the neighborhood conditional histogram is proposed to avoid the over-enhancement caused by the original histogram. When extracting the neighborhood conditional histogram of the input image, the neighbor content of each pixel is taken into consideration. In this way, the gray levels of the pixels located in the detail regions will get a high probability distribution function and will, accordingly, get a high contrast enhancement ratio, and vice versa, if the pixels are located in homogeneous regions. Then, for the improvement of CLAHE, we replace the clip-redistributed histogram with the proposed neighborhood conditional histogram. And to eliminate the block artifacts, the mapping function of each sub-block is updated based on the global mapping function. Lastly, the local contrast is optimized and enhanced based on both global and local enhanced result. The remainder of this paper is organized as follows. In Section 2, we first review and discuss the CLAHE. In Section 3, the proposed method is described in detail. In Section 4, experiments are conducted, and the analysis of the experimental results are given. In Section 5, the conclusion is drawn.

2. Review of CLAHE

CLAHE is an extension of the adaptive histogram equalization (AHE) algorithm [34]. These two methods are dedicated to solving the over-enhancement problem of HE. Over-enhancement is caused by the inherent drawback of HE. For a given infrared image X with L gray levels, the histogram can be expressed as
H ( l ) = n l ,   l = 0 , 1 , , L 1 ,
where n l indicates the number of pixels with a gray level l in the image. Based on the definition of HE and assuming that the dynamic range of the enhanced result is R (256 for the 8-bit images), the enhanced result of the gray level l is calculated by
y l = T ( l ) = R 1 N k = 0 l H ( k ) ,
where T ( l ) is the mapping function and maps the gray level l of the input image into y l of the output image, N is the total number of pixels in X . Therefore, for the enhanced image, the contrast enhancement ratio of the adjacent gray levels in the input image is
δ l = y l y l 1 = R 1 N H ( l ) .
As can be seen in Equation (3), the value of δ l is proportional to the corresponding histogram value. Which means, for the HE method, the gray level with a high histogram value also has a high enhancement ratio and occupies most grayscale range [7]. Unfortunately, the gray levels with histogram values usually belong to the pixels in the background or large homogeneous area. For these gray levels, high enhancement ratios and grayscale ranges result in unnaturalness and wash-out appearance, and the noise in the homogeneous regions will be amplified. At the same time, the other gray levels belonging to the pixels in the detail area cannot get desired enhancement ratios and grayscale ranges. This is the reason why HE usually causes over-enhancement for infrared images. In CLAHE, to avoid the over-enhancement of the homogeneous regions and get better contrast enhancement in detail regions, the image is divided into a grid of rectangular contextual sub-blocks [21], in each sub-block, HE is carried out. For each sub-block, H i , j is used to represent the histogram of the sub-block ( i , j ) . To avoid over-enhancement, H i , j is adjusted by clipping and redistribution [35] to get H ˜ i , j . The mechanism of this adjustment is shown in Figure 1. Where l i , j min and l i , j max are the minimum and maximum gray levels within the sub-block, respectively. The pixels in the green area of H i , j are redistributed to the blue area, thus H ˜ i , j is consists of the blue area and the part of H i , j below clip limit intuitively. This adjustment process is implemented by iteration [35].
As can be seen from the previous discussion, it is obvious that the enhancement ratio is related to the value of the clip limit. That is, a higher value of the clip limit results in more contrast enhancement of the corresponding gray levels. The clip limit of the original CLAHE method is a pre-defined value [21,36] and is invariant to the image content
c l i p = N b L + α ( N b N b L ) ,
where N b is the number of pixels in each sub-block, α is the clip factor within range [ 0 , 1 ] . Based on Equation (4), the range of clip limit is [ N b / L , N b ] . When the clip limit is equal to N b / L , the redistributed histogram H ˜ i , j turns to be a uniformly distributed histogram H ˜ i , j ( l ) = N b / L , which linearly maps the input image of range [ 0 , L 1 ] to the output image of range [ 0 , R 1 ] . Whereas, if the clip limit is higher than the histogram peak, the redistributed histogram H ˜ i , j is equal to the original histogram H i , j . It can also be observed that the clip limit determined by Equation (4) is the same for each sub-block. Based on H ˜ i , j , the mapping function of each sub-block is computed by
T i , j ( l ) = R 1 N b k = 0 l H ˜ i , j ( k ) ,   l = 0 , 1 , , L 1 ,
where T i , j ( l ) is the mapping function of the sub-block ( i , j ) . Due to the calculation of mapping functions being independent of each other, the block artifacts are inevitable. Therefore, to prevent the block artifacts, the mapping result of each pixel should take the mapping functions of the nearest four sub-blocks into consideration. For an arbitrary pixel p , the final remapped result is computed based on the bilinear interpolation [21] and is expressed as:
T ( X p ) = d 3 d 4 T 1 ( X p ) + d 1 d 4 T 2 ( X p ) + d 1 d 2 T 3 ( X p ) + d 2 d 3 T 4 ( X p ) ( d 1 + d 3 ) ( d 2 + d 4 ) ,
where X p is the gray value of the pixel p in the input image, T is an intermediate variable, T 1 ( ) , T 2 ( ) , T 3 ( ) , and T 4 ( ) are the mapping functions of the corresponding sub-blocks centered on O 1 , O 2 , O 3 , and O 4 as shown in Figure 2. Take an infrared image as an example, the HE result and CLAHE result of the given image are given in Figure 3. Here the resolution of the given image is 320 × 256 , and the size of each sub-block is 64 × 64 . The clip limit of CLAHE is computed by Equation (4), and we set α = 0.01 for the given image.
As shown in Figure 3, the contrast of the detail regions is not well enhanced in the HE result, and the CLAHE performs much better on contrast enhancement in these regions. However, in the homogeneous regions, the noise is over-enhancement and artifacts emerge in the CLAHE result. As discussed before, for the HE result, the contrast enhancement ratio of each gray level in the input image is proportional to the corresponding histogram value. Thus for the CLAHE, the maximum contrast enhancement ratio of each sub-block is determined by the clip limit. The clip limit is computed based on Equation (4) and is unrelated to the content of each sub-block. As a matter of fact, a higher clip limit is required for the sub-blocks with texture and details to get effective contrast enhancement. On the contrary, the sub-blocks with concentrated gray level distribution need a relatively lower clip limit to avoid the enhancement of noise. It is obvious that a constant clip limit is not suitable for CLAHE. In [37], the researchers introduced a mechanism to adaptively set the clip limit based on the content of each sub-block. This mechanism works for the 8-bit visual images, but it is not applicable to the captured 14-bit infrared images due to its being useless in the noise suppression of homogeneous regions. In addition to the noise amplification, the block artifacts are not completely eliminated in the homogeneous regions, as shown in Figure 3b. This is caused by the very different mapping results when applying the nearest four mapping function to the same gray levels if there are different proportions of homogeneous regions in the sub-blocks. Hence, the block artifacts still exist in the bilinear interpolation results.

3. Proposed Method

To avoid over-enhancement of homogeneous regions and effectively enhance the contrast of detail-rich regions, we replace the clip-redistributed histogram with our proposed neighborhood conditional histogram. In our proposed neighborhood conditional histogram, the image content is taken into consideration when computing the histogram. Therefore, the corresponding mapping function of each sub-block is able to automatically adapt to the content of each sub-block. Then, for the further elimination of block artifacts, the local mapping functions are updated based on the global mapping function. Lastly, the optimized local contrast enhanced result is obtained by making an optimization between the local Weber contrast of global and local enhanced results. In the following part, our proposed method is described in detail.

3.1. Neighborhood Conditional Histogram

As discussed in the previous section, the contrast enhancement ratio of each gray level after HE is proportional to the corresponding histogram value. In HE, for the pixels in homogeneous regions, the corresponding gray levels tend to get higher histogram values than other gray levels. Thus, this usually leads to the over-enhancement in the homogeneous regions and lack-enhancement in the detail regions. To address this problem, we try to take the image content into consideration when performing the histogram statistic. In our proposed neighborhood conditional histogram, the gray levels located in detail regions get higher histogram values than those located in homogeneous regions. The neighborhood conditional histogram of X is expressed as
H ( l ) = a = 1 A b = 1 B [ 1 w r 1 i = r r j = r r ϕ l ( X a , b , X a + i , b + j , t ) ] ,
where A × B represents the size of the given image, r is an integer introduced to determine the radius of the square neighborhood around each pixel, w r = ( 2 r + 1 ) 2 is the number of pixels in the square neighborhood, t is a predefined threshold, ϕ l ( u , v , t ) is defined to be a binary function and is expressed as follows:
ϕ l ( u , v , t ) = { 1 i f   u = l   a n d   | u v | t 0 o t h e r w i s e .
By defining the binary function for each pixel and its neighbors, the image content is able to be taken into consideration when computing the histogram value. Thus, a higher value of any gray level in our proposed neighborhood conditional histogram does not mean that the gray level has a larger probability of occurrence in the image. Instead, a higher value represents that the pixels that take on the corresponding gray level have a larger probability of being located in the detail regions. For the gray levels with a higher histogram value a better contrast enhancement after HE can be achieved. Hence, our proposed neighborhood conditional histogram is more reasonable than the original histogram. Based on Equation (2), the mapping function of the neighborhood conditional histogram can be obtained. The enhanced result of the given infrared image based on our proposed neighborhood conditional histogram is given in Figure 4. The parameters are set as r = 2 and t = 10 here for the given 14-bit infrared image. It is obvious that the neighborhood conditional histogram in Figure 4a looks much different from the histogram in Figure 3c. And the enhanced result looks much better than the original HE result.

3.2. Improved CLAHE

As discussed in the previous sub-section, our proposed neighborhood conditional histogram outperforms the original histogram on adaptively enhancing the contrast of infrared images. Therefore, for the improvement of CLAHE, we propose to replace the clip-redistributed histogram with the neighborhood conditional histogram. By applying the neighborhood conditional histogram to each sub-block, the new histogram of sub-block ( i , j ) is obtained and denoted as H i , j . And then the corresponding local mapping function T i , j ( l ) is computed by replacing H ˜ i , j ( k ) in Equation (5) with H i , j ( k ) . Obviously, if the mapping function T i , j ( l ) is applied to the sub-block ( i , j ) , the artifacts will still appear in the homogeneous regions after the bilinear interpolation. With this consideration, the local mapping functions have to be updated. As for the sub-blocks of the given infrared image, an indisputable fact is that each sub-block is just a part of the whole image. Thus, not only the mapping function of each sub-block but also the global mapping function have to be taken into consideration to update the local mapping functions. The local mapping functions are calculated based on the image content of the corresponding sub-blocks. Due to the calculation processes of the local mapping functions being carried out independently, the contrast of each sub-block can be well enhanced. However, the block artifacts may appear in the final enhanced result. On the contrary, the global mapping function is computed based on the whole image content. Therefore, there are no artifacts in the enhanced result, but the local contrast enhancement is not as good as the enhanced result using the local mapping functions. Considering the complementary characteristics of the global mapping function and the local mapping functions, we update the local mapping functions by making an adaptive compromise between the local mapping function and the global mapping function.
The global mapping function T G ( l ) is obtained by substituting H G ( l ) into Equation (2). Here H G ( l ) represents the neighborhood conditional histogram of the whole infrared image and is computed by adding the neighborhood conditional histograms of the sub-blocks together.
H G ( l ) = i j H i , j ( l ) .
The updated local mapping function T ˜ i , j is treated as the solution to a bi-criteria optimization problem [33]. We use the squared of the Euclidean norm for optimization. Hence, the updated mapping function is computed by
T ˜ i , j = arg min T T T G 2 2 + ƛ T T i , j 2 2 ,
where T is an intermediate variable, ƛ is a regularization parameter and varies over [ 0 , ) . The solution to this quadratic optimization problem is
T ˜ i , j = ( 1 β ) T G + β T i , j ,
where
β = ƛ 1 + ƛ .
As ƛ varies over [ 0 , ) , the value of β is within the range of [ 0 , 1 ] . It is obvious that the updated local mapping function T ˜ i , j turns out to be a weighted average of T G and T i , j . By varying β , the weights of T G and T i , j in T ˜ i , j change accordingly. When β = 0 , T ˜ i , j is equal to the global mapping function T G . Thus the enhanced result of sub-block ( i , j ) using T ˜ i , j is the same as the global enhanced result using T G . As β increases and approaches 1 , T ˜ i , j gradually converges to the original local mapping function T i , j and the corresponding enhanced result of sub-block ( i , j ) turns to resemble the enhanced result using T i , j . Therefore, β should be set as a block adaptive variable that has a different value depending on the block characteristics. This means it should be represented as the component of the sub-block ordered set ( β i , j β ) instead of the constant β . Considering this notation, the solution in Equation (11) is rewritten as
T ˜ i , j = ( 1 β i , j ) T G + β i , j T i , j .
As β i , j in Equation (12) is recommended to be a block adaptive value, it should be determined based on the block content. For the sub-blocks with a large proportion of homogeneous regions, a small β i , j is required to avoid the over-enhancement and the appearance of block artifacts. Whereas, a large β i , j is more suitable for the sub-blocks that contain many details and textures to get better contrast enhancement. When determining the value of β i , j , it is reasonable to utilize the neighborhood conditional histogram H i , j , which is also calculated based on the content of the sub-block ( i , j ) . Therefore, in our proposed method, β i , j is determined by
β i , j = l = 0 L 1 H i , j ( l ) N b .
From the definition of H i , j in Equation (7), for the sub-block ( i , j ) , a higher value of H i , j indicates that there are more pixels with gray level l located in the detail regions. This means that the sum of H i , j is larger if there are more details in the sub-block ( i , j ) , and accordingly, the value of β i , j is also larger as required. On the contrary, the value of β i , j for the sub-block with more homogeneous regions is also smaller. This is reflected in Figure 5, where it is clear that the homogeneous area accounts for a large proportion in sub-block 1 and sub-block 2. Therefore, the corresponding mapping functions resemble the global mapping function. As shown in Figure 5a, there are no artifacts in the homogeneous regions. This indicates that the improved CLAHE performs well on enhancing the local contrast with no artifacts.

3.3. Optimized Local Contrast Enhancement

To further enhance the local contrast and make the details more visible to human viewers, the optimized enhanced result is obtained by combining the local contrast of both global and local enhanced results. The global enhanced result Y G is computed by transferring the gray levels of the input infrared image to the output gray levels using the global mapping function T G . The local enhanced result Y L is obtained by the improved CLAHE described in the previous sub-section. With reference to a given pixel of the desired enhanced result Y , the local Weber contrast is defined as
C k = Y k Y ¯ k Y ¯ k ,
where C k denotes the Weber contrast of the pixel k , Y ¯ k represents the average gray intensity in the neighborhood of pixel k and is computed by
Y ¯ k = 1 | D k | i D k Y i ,
where D k represents the neighborhood of the pixel k and | D k | is the total number of pixels in the neighborhood. To enhance the local contrast, we suppose that the estimated local contrast of Y is expressed as
C ˜ k = c × C k ,
where C ˜ k is the estimated local contrast, c is defined as the contrast parameter, C k is the baseline local contrast. Taking the local contrast of both global and local enhanced results into consideration, the baseline local contrast C k is computed by solving a bi-criteria optimization problem as follows:
C k = arg min C C C G k 2 + λ C C L k 2 ,
where C is an intermediate variable, λ is defined as a regularization parameter and varies over [ 0 , ) , C G k , and C L k , representing the local contrast of the global and local enhanced results, respectively, are calculated by C G k = ( Y G k Y ¯ G k ) / Y ¯ G k and C L k = ( Y L k Y ¯ L k ) / Y ¯ L k . The optimal solution of Equation (17) is expressed as
C k = 1 1 + λ C G k + λ 1 + λ C L k .
As λ varies from 0 to , the solution of C k in Equation (18) traces the optimal trade-off local contrast between C G k and C L k . The baseline local contrast obtained by λ = 0 corresponds to the contrast of global enhanced result, whereas when λ increases to infinity, it resembles the contrast of the local enhanced result. Thus, we obtain different baseline local contrast according to λ . Taking the region characteristics of the input image into consideration, it is more reasonable to set λ as a region adaptive variable instead of a constant value. This means λ should be set as the component of the lexicographically ordered image ( λ k λ ). In the case of the contrast parameter c in Equation (16), the enhancement level of the estimated local contrast C ˜ k is adjusted by the value of c . To adaptively enhance the local contrast, in the proposed method, c is also determined to be a region adaptive variable by considering the regional characteristics. Therefore, the contrast parameter c should also be set as the component of the lexicographically ordered image ( c k c ) instead of a constant. Considering these notations and substituting Equation (18) into Equation (16), the estimated local contrast C ˜ k is rewritten as
C ˜ k = c k ( 1 1 + λ k C G k + λ k 1 + λ k C L k ) .
Combining Equations (14) and (19), the desired enhanced result Y k meets the formula below
Y k Y ¯ k Y ¯ k = c k ( 1 1 + λ k Y G k Y ¯ G k Y ¯ G k + λ k 1 + λ k Y L k Y ¯ L k Y ¯ L k ) ,
where Y ¯ k indicated the average gray intensity of the pixel k in the desired enhanced result, but the value is not given. To compute the desired enhanced result based on Equation (20), we assume that there is no significant difference between the average gray intensity of the local enhanced result and that of the desired enhanced result ( Y ¯ k Y ¯ L k ). Then, the desired enhanced result Y k is computed by
Y k = Y ¯ L k + c k ( 1 1 + λ k Y ¯ L k Y ¯ G k Y G k + λ k 1 + λ k Y L k Y ¯ L k ) .
As formulated in Equation (21), the desired enhanced result is determined by a constant term and a variable term. The first term has no contribution to the local contrast enhancement. Therefore, the local contrast is enhanced by determining the parameters λ k and c k adaptively. As discussed before, both λ k and c k should be set as region adaptive parameters that have different values depending on the regional characteristics of the input image. In our proposed method, the edge information of the input infrared image is utilized. The expressions for λ k and c k are given by
{ λ k = λ 0 Δ k c k = 1 + c 0 λ k / ( c 0 + λ k ) ,
where λ 0 and c 0 are user-defined parameters, Δ k is the edge information of the input image at the pixel k . In this paper, the edge information is computed by applying the Sobel operator to the input infrared image and then is linearly stretched to the same dynamic range of the enhanced result. It is clear that the values of λ k and c k are positively correlated with the edge information Δ k , and the range of the contrast parameter c k is [ 1 , 1 + c 0 ] . For the regions with sufficient edge information, the baseline local contrast resembles that of the local enhanced result and will further get enhanced, the extent of contrast enhancement is controlled by 1 + c 0 . On the contrary, if the regions have insufficient edge information, the baseline local contrast tends to be the same as that of global enhanced result and will later remain unchanged as c k = 1 . In our experiments, we empirically set λ 0 = 0.5 and c 0 = 1 for the 14-bit infrared images. A pseudocode of the proposed method is given as follows.
Algorithm 1: Adaptive contrast enhancement
Input: original infrared image X
1. Divide the image X into multiple equal-sized non-overlapped sub-blocks
2. For each sub-block, extract the neighborhood conditional histogram H i , j based on Equation (7)
3. Obtain the global mapping function T G and the original local mapping function T i , j based on H i , j
4. Compute the updated local mapping function T ˜ i , j based on T G and T i , j
5. Obtain improved CLAHE result Y L based on T ˜ i , j
6. Map the image X with T G to get the global enhanced result Y G
7. Enhance the local contrast combining Y L and Y G
Output: Enhanced result Y

4. Experimental Results

In this section, we apply the proposed method to six captured 14-bit infrared images and two raw infrared images extracted from the video sequences in BU-TIV [38] to evaluate its performance. The imager we used to capture the six infrared images is developed by our lab based on the sensor MARS-LW-RM3 with resolution 320 × 256 and is sensitive to the long-wave band 8 10   μ m . The resolution of the other two raw infrared images is 1024 × 512 and 512 × 512 . Figure 6 shows the HE results of the eight infrared images as a reference. In our experiments, the results are compared with those of four other block-based well-designed contrast enhancement methods: CLAHE, BCCE, ABMHE, and LGGSF. All of the experiments are conducted using MATLAB on a PC with Intel(R) Core(TM) i5 CPU (3.2 GHz) and 16GB RAM on a Windows 7 operating system.

4.1. The setting of Block Size and Threshold t

In this section, we conduct a further discussion on the block size and the threshold t . The block size is introduced to determine the size of each block in CLAHE, and the threshold t is introduced to compute the neighborhood conditional histogram in Equation (7). The proposed method is applied to a test image extracted from one image in BU-TIV with the resolution 320 × 256 to analyze the influence of different parameter settings to the enhanced result. The enhanced results with different block size and the threshold t are given in Figure 7. It can be seen that the nonuniformity of the test image is also enhanced when we set t = 10 . As t increases, the nonuniformity of the enhanced results is gradually suppressed, and some tiny details are also lost. Therefore, when determining the value of the threshold t , we should make a compromise between noise suppression and tiny detail preservation. For the block size, some artifacts appear in the enhanced result when the size of each block is set to be 16 × 16 . As the block size increases, the local contrast enhancement is not as good as that obtained by small block size, but this contrast enhancement reduction is not obvious when the threshold t is set to be a large value. In our experiments, we empirically set the block size to be 64 × 64 . For other infrared images with different resolution, the block size can also be set as 64 × 64 or another value, but not too small. For the six test images we captured, the threshold is set as t = 10 to get better local contrast enhancement. And for the other test images from BU-TIV, we set t = 20 for the suppression of nonuniformity.

4.2. Qualitative Evaluation

The experimental results of the infrared images are given in Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13. The results are obtained by applying CLAHE, BCCE, ABMHE, LGGSF, and the proposed method to the infrared images. In addition to these, the intermediate enhanced results of the proposed method, and the enhanced results of the improved CLAHE are given together for comparison. The block size in CLAHE, BCCE, ABMHE, and the proposed method is set to be 64 × 64 . The number of divisions in horizontal and vertical of the image for LGGSF is set to be k x = k y = 7 according to the original paper. In CLAHE and BCCE, the clip factor is set as α = 0.01 . In addition, the local contrast enhancement factors for BCCE and LGGSF are set as c = 8 . Lastly, for ABMHE, the step size of the partially overlapped in horizontal and vertical is set to be one-quarter of the block size. For good enhancement results, they should obtain better local contrast enhancement and noise suppression, be artifact-free and visually pleasing.
As shown in Figure 8, the enhanced results of CLAHE, BCCE, ABMHE, LGGSF, and the proposed method appear similar with respect to the building. CLAHE and BCCE cause noise amplification in the sky, which greatly reduces their performance compared with the other methods. ABMHE and LGGSF lost some details, as shown in the region marked by the red rectangle. This is caused by the improper classification in the transition region between the detail region and homogeneous region. The partial boundary of the mountain disappears in the ABMHE result, and some details of the trees in the LGGSF result also lost. All these undesired performance reductions do not appear in the enhanced result of GLHS and the proposed method. However, it is clear that the proposed method gets better local contrast enhancement than GLHS.
In Figure 9, the undesired noise amplification still appears in the enhanced result of CLAHE and BCCE. The details of the hills and buildings in BCCE result look clearer than those in the CLAHE result. Some part of the hills in the region marked by the red rectangle in the enhanced result of ABMHE cannot be distinguished from the sky. In the enhanced result of LGGSF, many details of the hills and the trees are smoothed. The local contrast enhancement of GLHS result is not as good as that of the other methods. Our proposed method produces a much better appearance and the finest details than the other methods.
In Figure 10, CLAHE and BCCE still enhance the noise in the region of the sky. ABMHE obtains relatively good enhanced results for there is no transition region between the building and the sky, which is the same situation as the experiments in the original paper. In the enhanced result of LGGSF, the details of the trees in the left and right of the subgraph are still smoothed. Once again, GLHS does not produce better local contrast enhancement, and the proposed method outperforms the other methods with no artifacts, good appearance, and fine details.
In Figure 11, the enhanced results of CLAHE and BCCE look unnatural due to the enhancement of noise and the appearance of block artifacts. As observed in the region marked by the red rectangle in the enhanced results of CLAHE, BCCE, and ABMHE, there are non-uniform brightness regions in the sky around the tower crane. These unpleasant appearance reductions cannot be found in the enhanced result of GLHS, LGGSF, and the proposed method. However, the details of the buildings and the tower crane do not look clearer than those of GLHS or the proposed method. In the homogeneous regions of the enhanced result of GLHS, some granular noise appears. The proposed produces an enhanced result with better visual quality improvement.
In Figure 12, CLAHE and BCCE produce better enhanced results than ABMHE, LGGSF, and GLHS, which benefits from the absence of the homogeneous region. As shown in the regions marked by the red rectangles in the enhanced results of ABMHE and LGGSF, many details of the trees are lost. This is caused by the improper classification of this low contrast region. The enhanced result of the proposed method looks better than that of BCCE, for the local contrast is well enhanced.
In Figure 13, the enhanced results of CLAHE, BCCE, ABMHE, LGGSF, and the proposed method resemble each other. The enhanced result of GLHS looks clear, but the local contrast is not well enhanced. Local contrast in the enhanced results of the proposed method and BCCE appears better than that of the other methods.
In Figure 14, due to the existence of nonuniformity, the enhanced results of CLAHE, BCCE, and LGGSF look poor because they are sensitive to the nonuniformity. In the enhanced result of ABMHE, there are non-uniform brightness regions in the road. GLHS performs best in terms of nonuniformity suppression. However, the local contrast is not well enhanced. The proposed method is able to suppress the nonuniformity and get better local contrast enhancement.
In Figure 15, the enhanced results of CLAHE and BCCE look terrible for the enhancement of the nonuniformity. The nonuniformity is well suppressed in the enhanced result of ABMHE, but there exists non-uniform brightness. The nonuniformity is suppressed in some regions of the LGGSF result. GLHS performs well on the nonuniformity suppression, but the local contrast is not well enhanced. The proposed method produces a better enhanced result with a natural appearance and nonuniformity suppression.

4.3. Quantitative Evaluation

In this section, to evaluate the performance of the six methods objectively, four metrics are adopted in this paper. They are the measure of enhancement by entropy (EMEE) [39], structural index (SI) [40], no-reference structural sharpness (NRSS) [41], and the lightness order error (LOE) [42].
EMEE is defined based on the Weber contrast and the concept of entropy, which is computed by the equation below.
E M E E ( I ) = 1 k 1 k 2 k = 1 k 1 l = 1 k 2 α [ I k , l max I k , l min + c ] α ln [ I k , l max I k , l min + c ] ,
where k 1 and k 2 are the number of divisions in horizontal and vertical of the given image I , I k , l max and I k , l min represent the maximum and minimum values of the block ( k , l ) , respectively, α is an additional parameter, and c is a small constant introduced to avoid dividing by zero. Generally, a larger EMEE value implies a higher contrast of the given image. In this paper, we set α = 0.2 , c = 0.0001 , and the block size is 8 × 8 .
SI is defined based on the correlation coefficient, and the expression is given below.
S I ( X , Y ) = σ X , Y + c σ X σ Y + c .
where X and Y represent the input image and the enhanced image, σ X , Y is the covariance of X and Y , σ X and σ Y are the standard deviation, c is a small constant introduced to avoid dividing by zero, and σ X , Y is computed by the equation below.
σ X , Y = 1 N 1 i = 1 N ( I X ( i ) μ I X ) ( I Y ( i ) μ I Y ) ,
where N is the number of pixels in the images, μ I X and μ I Y are, respectively, the mean intensity of I X and I Y . It is clear that SI indicates the global structural similarity of the enhanced image when compared to the input image. A larger SI value means that there is a high similarity between the enhanced image and the input image in terms of structure.
NRSS is defined based on the well-known SSIM (structural similarity index measurement) [43]. It indicates the structural sharpness of the given image and is computed by the equation below.
N R S S ( I ) = 1 1 K i = 1 K S S I M ( B i G I , B i G F ( I ) ) ,
where B i G I and B i G F ( I ) are one of the K overlapped sub-blocks with higher variations in G I and G F ( I ) , G I represents the gradient image and is computed by applying the Sobel operator to the given image I , and G F ( I ) is the gradient image of the Gaussian-blurred image F ( I ) . A large NRSS value means that the given image looks clear with high contrast. In this paper, the block size in NRSS is set to be 32 × 32 , the step size is set to be half of the block size, and K is equal to half of the total number of the overlapped sub-blocks.
LOE is defined to indicate the lightness order error between the input image and the enhanced image. The expression of LOE is given below.
{ L O E = 1 A B a = 1 A b = 1 B R D a , b R D a , b = m = 1 A n = 1 B [ U ( X a , b D , X m , n D ) U ( Y a , b D , Y m , n D ) ] U ( x , y ) = { 1 , f o r   x y 0 , e l s e ,
where A × B represents the image size of X D and Y D , U ( x , y ) is the unit step function, is the exclusive or operator, X D and Y D are the down-sampled versions of X and Y to reduce the computational complexity. The ratio between the size of the input image and that of the down-sampled image is set as r = 16 in this paper. Obviously, a small LOE value indicates that the enhanced image gets better naturalness preservation of the input image.
In Table 1, the quantitative evaluation results of the six methods in terms of the four metrics are given. The highest results of EMEE, SI, and NRSS, the lowest results of LOE are marked in bold in the table. Among these methods, the other five methods except GLHS are block-based methods, and the foundation of GLHS is global histogram specification. Therefore, GLHS usually gets the highest SI values and the lowest LOE values. The proposed method provides almost the highest EMEE values, except in Hills. Although the EMEE value of the proposed method in Hills is slightly smaller than that of ABMHE, the proposed method provides the highest average EMEE value among the five methods. For the SI values, due to the utilization of block-based local contrast enhancement, the SI values of the five block-based methods are lower than those of GLHS. The proposed method usually gives relatively higher SI values among the block-based methods, and the average SI value of the proposed method is the highest in the five block-based methods. This means the proposed method performs well on the structural similarity with the input image. Benefitting from the local contrast enhancement process, the NRSS values of GLHS are the highest among the six methods. For the first six infrared images, the proposed method usually produces relatively high NRSS values. Whereas for the other two test infrared images, the NRSS values of the proposed method are not as high as those of BCCE and LGGSF, this is caused by the setting of the higher threshold. The reason why the NRSS values are not the highest among the methods is the combination of both global and local enhanced results when enhancing the local contrast. Comparison of LOE values indicates that the proposed method almost provides the lowest values among the five block-based methods, except in Sky and Trees. However, the average LOE value of the proposed method is obviously the lowest among that of the other block-based methods. In summary, the values of the four metrics demonstrate that the proposed method outperforms the other five methods in terms of local contrast enhancement and performs well on the structural similarity, detail improvement, and naturalness preservation.
In Table 2, the average time consumption for the six methods to process the six captured test images are given. It is clear that the time computation of CLAHE and BCCE is lower than those of the other three methods. The local contrast enhancement process of BCCE leads to a higher computation time than CLAHE. Due to the utilization of sub-block classification and labeling, LGGSF needs more computation time than BCCE. The computation time of the proposed method is higher than LGGSF and BCCE, this is caused by the computation of neighborhood conditional histogram in each sub-block and the optimized contrast enhancement process. For the GLHS, the utilization of the 2D histogram, histogram specification, and the local contrast enhancement process lead to a more time-consuming process. In ABMHE, the large amount of overlapped sub-blocks and the sub-block searching operations make it a computationally expensive method. Therefore, its consumption time is the highest among the six methods.

5. Conclusions

In this paper, to improve the visual quality of infrared images, we proposed an adaptive contrast enhancement method based on the neighborhood conditional histogram. We introduced the neighborhood conditional histogram into CLAHE to avoid over-enhancement and achieve adaptive contrast enhancement for various sub-blocks. To eliminate the block-artifacts, the mapping functions are further updated based on the global mapping function. Lastly, the local contrast is optimized based on both global and local enhanced result, and then adaptively enhanced based on the edge information of the input image. The neighborhood conditional histogram and the strategy that combines both global and local mapping functions to reduce the block artifacts and enhance the local contrast are the main innovations of this paper. By setting different parameters for the neighborhood conditional, the proposed method is able to adaptively enhance the local contrast and suppress the nonuniformity of the input infrared images. In addition, the proposed method has the potential to be implemented on the hardware platform, such as FPGA. Therefore, the proposed method can be applied to the infrared imaging system to improve performance.

Author Contributions

C.L. proposed the concept, performed the method, analyzed the data, conducted the experiments and wrote the paper; X.S. acted as the research supervisor; X.K. and Y.L. helped modify the language expression; G.G. and Q.C. provided technical assistance to the research; The manuscript was finally discussed by all the co-authors.

Funding

This paper is financially supported by the National Natural Science Foundation of China (China, Grant no. 11503010, 11773018), the Fundamental Research Funds for the Central Universities (China, Grant no. 30916015103), Qing Lan Project and Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging & Intelligence Sense (China, Grant no. 3091601410405).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Weng, Q. Thermal infrared remote sensing for urban climate and environmental studies: Methods, applications, and trends. ISPRS J. Photogramm. Remote Sens. 2009, 64, 335–344. [Google Scholar] [CrossRef]
  2. Cao, Y.; Yang, M.Y.; Tisse, C.L. Effective Strip Noise Removal for Low-Textured Infrared Images Based on 1-D Guided Filtering. IEEE Trans. Circuits Syst. Video Technol. 2016, 26, 2176–2188. [Google Scholar] [CrossRef]
  3. Ring, E.; Ammer, K. Infrared thermal imaging in medicine. Physiol. Meas. 2012, 33, R33. [Google Scholar] [CrossRef]
  4. Liu, C.; Sui, X.; Liu, Y.; Kuang, X.; Gu, G. FPN estimation based nonuniformity correction for infrared imaging system. Infrared Phys. Technol. 2018, 96, 22–29. [Google Scholar] [CrossRef]
  5. Sun, X.; Liu, X.; Tang, Z.; Long, G.; Yu, Q. Real-time visual enhancement for infrared small dim targets in video. Infrared Phys. Technol. 2017, 83, 217–226. [Google Scholar] [CrossRef]
  6. Kim, J.-H.; Kim, J.-H.; Jung, S.-W.; Ko, S.-J.; Noh, C.-K. Novel contrast enhancement scheme for infrared image using detail-preserving stretching. OPTICE 2011, 50, 077002. [Google Scholar] [CrossRef]
  7. Huang, J.; Ma, Y.; Zhang, Y.; Fan, F. Infrared image enhancement algorithm based on adaptive histogram segmentation. Appl. Opt. 2017, 56, 9686–9697. [Google Scholar] [CrossRef] [PubMed]
  8. Gonzalez, R.C.; Woods, R.E.; Eddins, S.L. Digital Image Processing Using MATLAB; Pearson-Prentice-Hall: Upper Saddle River, NJ, USA, 2004; Volume 624. [Google Scholar]
  9. Song, K.S.; Kang, M.G. Optimized Tone Mapping Function for Contrast Enhancement considering Human Visual Perception System. IEEE Trans. Circuits Syst. Video Technol. 2018. [Google Scholar] [CrossRef]
  10. Lin, C.-L. An approach to adaptive infrared image enhancement for long-range surveillance. Infrared Phys. Technol. 2011, 54, 84–91. [Google Scholar] [CrossRef]
  11. Vickers, V.E. Plateau equalization algorithm for real-time display of high-quality infrared imagery. OPTICE 1996, 35, 1921–1927. [Google Scholar] [CrossRef]
  12. Wang, B.-J.; Liu, S.-Q.; Li, Q.; Zhou, H.-X. A real-time contrast enhancement algorithm for infrared images based on plateau histogram. Infrared Phys. Technol. 2006, 48, 77–82. [Google Scholar] [CrossRef]
  13. Liang, K.; Ma, Y.; Xie, Y.; Zhou, B.; Wang, R. A new adaptive contrast enhancement algorithm for infrared images based on double plateaus histogram equalization. Infrared Phys. Technol. 2012, 55, 309–315. [Google Scholar] [CrossRef]
  14. Li, S.; Jin, W.; Li, L.; Li, Y. An improved contrast enhancement algorithm for infrared images based on adaptive double plateaus histogram equalization. Infrared Phys. Technol. 2018, 90, 164–174. [Google Scholar] [CrossRef]
  15. Kim, Y.-T. Contrast enhancement using brightness preserving bi-histogram equalization. IEEE Trans. Consum. Electron. 1997, 43, 1–8. [Google Scholar]
  16. Wang, Y.; Chen, Q.; Zhang, B. Image enhancement based on equal area dualistic sub-image histogram equalization method. IEEE Trans. Consum. Electron. 1999, 45, 68–75. [Google Scholar] [CrossRef]
  17. Chen, S.-D.; Ramli, A.R. Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation. IEEE Trans. Consum. Electron. 2003, 49, 1301–1309. [Google Scholar] [CrossRef]
  18. Wan, M.; Gu, G.; Qian, W.; Ren, K.; Chen, Q.; Maldague, X. Infrared Image Enhancement Using Adaptive Histogram Partition and Brightness Correction. Remote Sens. 2018, 10, 682. [Google Scholar] [CrossRef]
  19. Arici, T.; Dikbas, S.; Altunbasak, Y. A histogram modification framework and its application for image contrast enhancement. IEEE Trans. Image Process. 2009, 18, 1921–1935. [Google Scholar] [CrossRef]
  20. Xiao, B.; Tang, H.; Jiang, Y.; Li, W.; Wang, G. Brightness and contrast controllable image enhancement based on histogram specification. Neurocomputing 2018, 275, 2798–2809. [Google Scholar] [CrossRef]
  21. Zuiderveld, K. Contrast limited adaptive histogram equalization. Graph. Gems 1994, 474–485. [Google Scholar]
  22. Kim, J.-Y.; Kim, L.-S.; Hwang, S.-H. An advanced contrast enhancement using partially overlapped sub-block histogram equalization. IEEE Trans. Circuits Syst. Video Technol. 2001, 11, 475–484. [Google Scholar]
  23. Branchitta, F.; Diani, M.; Corsini, G.; Porta, A. Dynamic-range compression and contrast enhancement in infrared imaging systems. OPTICE 2008, 47, 076401. [Google Scholar] [CrossRef]
  24. Wang, Y.; Pan, Z. Image contrast enhancement using adjacent-blocks-based modification for local histogram equalization. Infrared Physics Technol. 2017, 86, 59–65. [Google Scholar] [CrossRef]
  25. Li, S.; Jin, W.; Wang, X.; Li, L.; Liu, M. Contrast Enhancement Algorithm for Outdoor Infrared Images based on Local Gradient-grayscale Statistical Feature. IEEE Access 2018, 6. [Google Scholar] [CrossRef]
  26. Yuan, L.T.; Swee, S.K.; Ping, T.C. Infrared image enhancement using adaptive trilateral contrast enhancement. Pattern Recognit. Lett. 2015, 54, 103–108. [Google Scholar] [CrossRef]
  27. Branchitta, F.; Diani, M.; Corsini, G.; Romagnoli, M. New technique for the visualization of high dynamic range infrared images. OPTICE 2009, 48, 096401. [Google Scholar] [CrossRef]
  28. Zuo, C.; Chen, Q.; Liu, N.; Ren, J.; Sui, X. Display and detail enhancement for high-dynamic-range infrared images. OPTICE 2011, 50, 127401. [Google Scholar] [CrossRef]
  29. Li, Y.; Hou, C.; Tian, F.; Yu, H.; Guo, L.; Xu, G.; Shen, X.; Yan, W. Enhancement of infrared image based on the retinex theory. In Proceedings of the 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France, 22–26 August 2007; pp. 3315–3318. [Google Scholar]
  30. Zhan, B.; Wu, Y. Infrared image enhancement based on wavelet transformation and retinex. In Proceedings of the 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics, Nanjing, China, 26–28 August 2010; pp. 313–316. [Google Scholar]
  31. Mallat, S.G. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 1989, 11, 674–693. [Google Scholar] [CrossRef]
  32. Singh, R.; Khare, A. Fusion of multimodal medical images using Daubechies complex wavelet transform–A multiresolution approach. Inf. Fusion 2014, 19, 49–60. [Google Scholar] [CrossRef]
  33. Liu, C.; Sui, X.; Kuang, X.; Liu, Y.; Gu, G.; Chen, Q. Optimized Contrast Enhancement for Infrared Images Based on Global and Local Histogram Specification. Remote Sens. 2019, 11, 849. [Google Scholar] [CrossRef]
  34. Pizer, S.M.; Amburn, E.P.; Austin, J.D.; Cromartie, R.; Geselowitz, A.; Greer, T.; ter Haar Romeny, B.; Zimmerman, J.B.; Zuiderveld, K. Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 1987, 39, 355–368. [Google Scholar] [CrossRef]
  35. Unal, B.; Akoglu, A. Resource efficient real-time processing of contrast limited adaptive histogram equalization. In Proceedings of the 2016 26th International Conference on Field Programmable Logic and Applications (FPL), Lausanne, Switzerland, 29 August–2 September 2016; pp. 1–8. [Google Scholar]
  36. Jenifer, S.; Parasuraman, S.; Kadirvelu, A. Contrast enhancement and brightness preserving of digital mammograms using fuzzy clipped contrast-limited adaptive histogram equalization algorithm. Appl. Soft Comput. 2016, 42, 167–177. [Google Scholar] [CrossRef]
  37. Chang, Y.; Jung, C.; Ke, P.; Song, H.; Hwang, J. Automatic Contrast Limited Adaptive Histogram Equalization with Dual Gamma Correction. IEEE Access 2018, 6, 11782–11792. [Google Scholar] [CrossRef]
  38. Wu, Z.; Fuller, N.; Theriault, D.; Betke, M. A thermal infrared video benchmark for visual analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA, 23–28 June 2014; pp. 201–208. [Google Scholar]
  39. Panetta, K.; Gao, C.; Agaian, S. No reference color image contrast and quality measures. IEEE Trans. Consum. Electron. 2013, 59, 643–651. [Google Scholar] [CrossRef]
  40. Rivera, A.R.; Ryu, B.; Chae, O. Content-aware dark image enhancement through channel division. IEEE Trans. Image Process. 2012, 21, 3967–3980. [Google Scholar] [CrossRef] [PubMed]
  41. Xie, X.; Zhou, J.; Wu, Q. No-reference quality index for image blur. J. Comput. Appl. 2010, 30, 921–924. [Google Scholar] [CrossRef]
  42. Wang, S.; Zheng, J.; Hu, H.-M.; Li, B. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 2013, 22, 3538–3548. [Google Scholar] [CrossRef]
  43. ZWang, Z. Image quality assessment from error measurement to structural similarity. IEEE Trans. Image Process. 2004, 13, 600r612. [Google Scholar]
Figure 1. Histogram adjustment within sub-block (i, j).
Figure 1. Histogram adjustment within sub-block (i, j).
Remotesensing 11 01381 g001
Figure 2. Diagram of the optimization problem.
Figure 2. Diagram of the optimization problem.
Remotesensing 11 01381 g002
Figure 3. The enhancement result of the given infrared image using contrast-limited adaptive histogram equalization (CLAHE). (a) The histogram equalization (HE) result of the given image. (b) The CLAHE result of the given image. (c) The histogram of the given infrared image. (d) The histogram of each sub-block and the red line represents the clip limit. (e) The mapping function of HE and the mapping functions of sub-block 1, 2, and 3 in CLAHE. The X-axis ranges in (ce) are set to be (9000,11,000) for better visualization.
Figure 3. The enhancement result of the given infrared image using contrast-limited adaptive histogram equalization (CLAHE). (a) The histogram equalization (HE) result of the given image. (b) The CLAHE result of the given image. (c) The histogram of the given infrared image. (d) The histogram of each sub-block and the red line represents the clip limit. (e) The mapping function of HE and the mapping functions of sub-block 1, 2, and 3 in CLAHE. The X-axis ranges in (ce) are set to be (9000,11,000) for better visualization.
Remotesensing 11 01381 g003
Figure 4. The enhanced result of the given infrared image. (a) The neighborhood conditional histogram of the given infrared image. (b) The corresponding enhanced result based on the neighborhood conditional histogram. The X-axis range in (a) is set to be (9000,11,000) for better visualization.
Figure 4. The enhanced result of the given infrared image. (a) The neighborhood conditional histogram of the given infrared image. (b) The corresponding enhanced result based on the neighborhood conditional histogram. The X-axis range in (a) is set to be (9000,11,000) for better visualization.
Remotesensing 11 01381 g004
Figure 5. The enhanced result of the given infrared image using the improved CLAHE. (a) The enhanced result. (b) The neighborhood conditional histogram of each sub-block and the red line is introduced to be a reference value of ten. (c) The global mapping function and the local mapping functions of sub-block 1, 2, and 3 in the improved CLAHE. The X-axis range in (b,c) are set to be (9000,11,000) for better visualization.
Figure 5. The enhanced result of the given infrared image using the improved CLAHE. (a) The enhanced result. (b) The neighborhood conditional histogram of each sub-block and the red line is introduced to be a reference value of ten. (c) The global mapping function and the local mapping functions of sub-block 1, 2, and 3 in the improved CLAHE. The X-axis range in (b,c) are set to be (9000,11,000) for better visualization.
Remotesensing 11 01381 g005
Figure 6. The HE results of the six infrared images used in the experiments. (a) Building1, (b) Hills, (c) Building2, (d) Sky, (e) Trees, (f) Windows, (g) Road, (h) Students.
Figure 6. The HE results of the six infrared images used in the experiments. (a) Building1, (b) Hills, (c) Building2, (d) Sky, (e) Trees, (f) Windows, (g) Road, (h) Students.
Remotesensing 11 01381 g006
Figure 7. The enhanced results of the proposed method with different parameter setting.
Figure 7. The enhanced results of the proposed method with different parameter setting.
Remotesensing 11 01381 g007
Figure 8. Enhancement results for Building1. (a) contrast-limited adaptive histogram equalization (CLAHE), (b) balanced CLAHE and contrast enhancement (BCCE), (c) adjacent-blocks-based modification for local histogram equalization (ABMHE), (d) local gradient-grayscale statistical feature (LGGSF), (e) global and local histogram specification (GLHS), (f) Proposed.
Figure 8. Enhancement results for Building1. (a) contrast-limited adaptive histogram equalization (CLAHE), (b) balanced CLAHE and contrast enhancement (BCCE), (c) adjacent-blocks-based modification for local histogram equalization (ABMHE), (d) local gradient-grayscale statistical feature (LGGSF), (e) global and local histogram specification (GLHS), (f) Proposed.
Remotesensing 11 01381 g008
Figure 9. Enhancement results for Hills. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Figure 9. Enhancement results for Hills. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Remotesensing 11 01381 g009
Figure 10. Enhancement results for Building2. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Figure 10. Enhancement results for Building2. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Remotesensing 11 01381 g010
Figure 11. Enhancement results for Sky. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Figure 11. Enhancement results for Sky. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Remotesensing 11 01381 g011
Figure 12. Enhancement results for Trees. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Figure 12. Enhancement results for Trees. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Remotesensing 11 01381 g012
Figure 13. Enhancement results for Windows. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Figure 13. Enhancement results for Windows. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Remotesensing 11 01381 g013
Figure 14. Enhancement results for Road. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Figure 14. Enhancement results for Road. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS, (f) Proposed.
Remotesensing 11 01381 g014
Figure 15. Enhancement results for Students. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS. (f) Proposed.
Figure 15. Enhancement results for Students. (a) CLAHE, (b) BCCE, (c) ABMHE, (d) LGGSF, (e) GLHS. (f) Proposed.
Remotesensing 11 01381 g015
Table 1. The quantitative evaluation results of contrast-limited adaptive histogram equalization (CLAHE), balanced CLAHE and contrast enhancement (BCCE), adjacent-blocks-based modification for local histogram equalization (ABMHE), local gradient-grayscale statistical feature (LGGSF), global and local histogram specification (GLHS), and the proposed method in terms of enhancement by entropy (EMEE), structural index (SI), no-reference structural sharpness (NRSS), and lightness order error (LOE).
Table 1. The quantitative evaluation results of contrast-limited adaptive histogram equalization (CLAHE), balanced CLAHE and contrast enhancement (BCCE), adjacent-blocks-based modification for local histogram equalization (ABMHE), local gradient-grayscale statistical feature (LGGSF), global and local histogram specification (GLHS), and the proposed method in terms of enhancement by entropy (EMEE), structural index (SI), no-reference structural sharpness (NRSS), and lightness order error (LOE).
MetricsMethodsInfrared Images in the ExperimentsAverage
Building1HillsBuilding2SkyTreesWindowsRoadStudents
EMEECLAHE0.20000.19630.57700.33170.48230.57531.49990.35920.5277
BCCE0.43880.44860.70910.64561.08991.51211.89790.48390.9032
ABMHE1.41801.81561.63001.86972.58991.49871.68590.72421.6540
LGGSF0.29600.21530.63600.14971.26460.94321.17860.17010.6067
GLHS0.38090.23772.87970.40551.57281.92950.52630.37301.0382
Proposed3.38921.80315.54962.16525.92574.87952.56740.87213.3940
SICLAHE0.68730.68510.68870.68850.65690.80210.64760.75860.7019
BCCE0.64490.63880.66720.59070.64760.79430.61970.55950.6453
ABMHE0.79300.85630.65410.91030.79240.81970.86760.82110.8143
LGGSF0.83470.92830.75370.97780.65750.83410.70310.85510.8180
GLHS0.95770.94940.86080.97930.90270.92110.97610.90940.9321
Proposed0.86070.89960.75030.96250.72470.84900.89990.95200.8623
NRSSCLAHE0.87040.84420.85350.84830.79600.76820.77340.81150.8207
BCCE0.92370.90380.90920.89810.86090.82020.87230.88060.8836
ABMHE0.85780.83430.84250.82630.77800.81410.76560.78560.8130
LGGSF0.89250.87970.87480.87700.84740.82570.84910.86840.8643
GLHS0.96130.95830.93980.93520.89970.90550.87710.88510.9203
Proposed0.91600.89880.91240.88790.86600.85580.81020.83490.8728
LOECLAHE73.8987.2764.9897.0970.7343.24523.51257.11152.23
BCCE79.9589.9668.28104.9274.4644.77544.24324.97166.44
ABMHE42.4641.4144.8351.3839.9048.97338.67241.06106.09
LGGSF34.6835.1535.389.6782.7441.20463.96201.55113.04
GLHS11.269.51569.088.7510.1310.3563.0428.0118.77
Proposed23.4726.4425.7817.5653.6429.90274.7694.6768.28
Table 2. The average time consumption of the six methods on the six captured test images (Unit: Second).
Table 2. The average time consumption of the six methods on the six captured test images (Unit: Second).
MethodCLAHEBCCEABMHELGGSFGLHSProposed
Time0.06750.08423.24710.15220.91720.1674

Share and Cite

MDPI and ACS Style

Liu, C.; Sui, X.; Kuang, X.; Liu, Y.; Gu, G.; Chen, Q. Adaptive Contrast Enhancement for Infrared Images Based on the Neighborhood Conditional Histogram. Remote Sens. 2019, 11, 1381. https://doi.org/10.3390/rs11111381

AMA Style

Liu C, Sui X, Kuang X, Liu Y, Gu G, Chen Q. Adaptive Contrast Enhancement for Infrared Images Based on the Neighborhood Conditional Histogram. Remote Sensing. 2019; 11(11):1381. https://doi.org/10.3390/rs11111381

Chicago/Turabian Style

Liu, Chengwei, Xiubao Sui, Xiaodong Kuang, Yuan Liu, Guohua Gu, and Qian Chen. 2019. "Adaptive Contrast Enhancement for Infrared Images Based on the Neighborhood Conditional Histogram" Remote Sensing 11, no. 11: 1381. https://doi.org/10.3390/rs11111381

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop