An Image Denoising Method for a Visible Light Camera in a Complex Sky-Based Background

Abstract

For space target images captured by a sky-based visible light camera, various conditions are influenced by the imaging system itself and the observation environment; these conditions include uneven image background intensity, complex noise, stray light composition, and diverse target forms. A mean wavelet transform algorithm is proposed. This algorithm initially performs mean filtering and wavelet transform filtering on the noise-containing space target images and then performs a wavelet inverse transform on the filtered results to generate the final image. The experimental results show that our algorithm has good denoising performance and can effectively maintain the image details.

1. Introduction

Studies on sky-based image denoising techniques have focused on extracting space targets by processing the image prior to object detection. The objective is to suppress or eliminate inhomogeneous background noise.

Methods for suppressing or eliminating image noise can be categorized into two groups: transform domain methods [1,2,3,4] and spatial domain methods [5,6,7,8,9,10,11,12,13]. The transform domain approach involves removing the noise in the transformed domain of the image and then inverting the transform to achieve image denoising. Spatial domain methods, on the other hand, involve directly operating on the original image, taking advantage of the grayscale values in the neighborhood of the pixels to perform relevant operations on the pixels.

Unlike conventional images, in space target images, the captured images not only contain Gaussian noise, salt-and-pepper noise, Poisson noise, speckle noise, etc., but also the imaging size, shape, and gray value of space targets are highly similar to noise, and the background strength is unevenly distributed due to the scattered light. These factors cause difficulty for traditional image denoising algorithms [14,15,16,17,18,19] to be applied to the denoising of star maps, easily causing loss of target information or false targets. New requirements for image processing technology are needed to suppress or eliminate the complex background interference in star maps while preserving space target information.

This paper proposes a mean curvelet transform method. Curvelet transform is a multiscale and directionally sensitive transformation method that extracts detailed features from an image. On the other hand, mean processing is a basic image smoothing method that reduces noise by calculating the average value of pixel neighborhoods. By integrating these two methods, one can first utilize the curvelet transform to extract detailed features from the image and then use mean processing to smooth the extracted details, thereby reducing noise. The effectiveness of the proposed algorithm is verified through simulation experiments.

2. Materials and Methods

2.1. Wavelet Transform and Curvelet Transform

2.1.1. Wavelet Transform

In the 1980s, J. Morlet first proposed the concept of wavelets [20]. In the same period, mathematicians Y. Meyer and S. Mallat further studied the Mallat algorithm based on previous work, and the wavelet transform was officially created [21,22]. From a theoretical perspective, wavelet transform theory evolved from Fourier analysis. Fourier analysis can be used to analyze the characteristics of data separately using the time axis and frequency axis transformations, but it cannot reflect the characteristics of signals changing simultaneously along both the time and frequency axes. It can only consider one of the two axes, allowing a global understanding of the signal’s characteristics, but is unable to describe the local characteristics of a specific time-frequency region of the signal. In reality, signals are often nonstationary, and it is difficult to maintain stability. Most signals are nonstationary, and the local time-frequency characteristics precisely describe the distinctive features of nonstationary signals. While scaling the signal by shrinking or expanding the variable multiplier and transforming the signal through smooth distance shifts, the wavelet transform can decompose the signal into multiple levels, surpassing the limitations of the Fourier transform in this aspect.

The traditional approach of the wavelet transform is to separate the high-frequency and low-frequency signals for processing. The low-frequency signal, which undergoes minimal changes, is expanded to analyze the fine details of the rapidly changing parts in the high-frequency signal. In the time dimension, the signal is further segmented, while in the low-frequency signal, the frequency is divided more finely, adjusting the time-frequency characteristics analysis based on the specific high and low frequencies of the signal.

The continuous wavelet transform involves extensive data operations, so it needs to be discretized during implementation to simplify the form. In the expression of continuous wavelets, the binary number scale parameter a and the translation parameter b remain continuous, and only the parameters in the semi-discrete continuous wavelet formula are discretized while the variable t remains unchanged. The discrete form of the wavelet transform is:

$$\left(W_{\mathsf{\Psi }}f\right)\left(\frac{1}{2^j},b\right)={\int }_{-\infty }^{\infty }f\left(t\right)\left\{2^{j/2}\mathsf{\Psi }\left(2^j(x-b)\right)\right\}dx$$

(1)

This transformation is also known as the binary wavelet transform, which meets the condition:

$$A\le {\left.\sum _{j\in z}\left(2^{-j}\omega \right)\right|}^2\le B$$

(2)

Here, $0<A<B<\infty$, where A and B are constants. For a certain level of decomposition, the binary wavelet transform can be considered a function of the variable b in Equation (2), and b is not discretized. Equation (2) can be transformed into a convolution operation expression, which is shown below:

$$W_{s}f\left(x\right)=f^{*}{\mathsf{\Psi }}_s\left(x\right)$$

(3)

By scaling the wavelet mother function $\mathsf{\Psi }\left(x\right)$ with a factor of s (where $s=2j$), we obtain the wavelet function as follows:

$${\mathsf{\Psi }}_s\left(x\right)=\frac{1}{s}{\mathsf{\Psi }}_s\left(\frac{x}{s}\right)$$

(4)

Let us assume $f_{\tau }\left(x\right)=f(x-\tau )$. Then,

$$\left(W_{2j}{f}_{\tau }\right)\left(x\right)=W_{2j}\left[f\left(x\right)\right]$$

(5)

If we first shift the function f along the scale and then perform the binary wavelet transform, it is equivalent to first applying the binary wavelet transform and then shifting it along the scale. In other words, $W_{2j}f$ possesses the properties of f, which is the meaning of translation invariance.

Although the binary wavelet transform is in a discrete form, the variable b is not discretized along the scale, and the transformation of the wavelet transform at different scales is not discrete. This translation invariance exhibits a zooming characteristic. By adjusting b to enlarge or reduce the scaling factor, the wavelet transform acts like a microscope, enabling the study of finer or coarser details in the signal.

2.1.2. Limitations of the Wavelet Transform and Introduction of the Ridgelet Transform

Although the wavelet transform, known as the “mathematical microscope”, can express the details of image waveforms and has significant application value and advantages, it has a limitation due to the lack of anisotropy in wavelet bases. The wavelet transform can only reflect discontinuities or points where the derivative is zero in 1D images, and it cannot capture the edge features of 2D spatial images. Therefore, at the end of the 20th century, Candes developed the ridgelet transform algorithm [23], which expresses line singularities and can provide a good approximation of line singularities in multivariable functions. The expression of the ridgelet transform is given by:

$${\mathsf{\Psi }}_{\gamma }\left(x\right)=a^{-\frac{1}{2}}\mathsf{\Psi }\left(\frac{[u,x]-b}{a}\right)$$

(6)

where ${\mathsf{\Psi }}_{\gamma }$ is called a ridgelet generated from $\mathsf{\Psi }$ under specific admissibility conditions. a represents the scale parameter, "u" represents the direction parameter, and b represents the position parameter. ${\mathsf{\Psi }}_{\gamma }$ has rapid decay properties that satisfy:

$$\int \mathsf{\Psi }(t)dt=0$$

(7)

For functions with several variables, the ridgelet transform is defined as:

$$R\left(f\right)\left(r\right)=〈f,{\mathsf{\Psi }}_r〉$$

(8)

The ridgelet transform represents the direction by adding an additional parameter to the wavelet basis function. In the cross-section, the basis of the ridge is the wavelet. The continuous ridgelet transform in 2D space is expressed in Equation (9).

$$CR{T}_f\left(a,b_x,b_y\right)={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }f(x,y)\mathsf{\Psi }a,b_x,b_y(x,y)dxdy$$

(9)

When comparing the expressions of the 2D wavelet transform, as shown in Equation (10), with the ridgelet transform, we can observe the following contrast: in the wavelet transform formula, the parameter is a point, whereas in the ridgelet transform, the parameter is a line.

$$w_f\left(a,b_x,b_y\right)={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }f(x,y)\mathsf{\Psi }a,b_x,b_y(x,y)dxdy$$

(10)

The introduction of the ridgelet transform solves the problem of the wavelet transform’s inability to describe 2D edge singularities. However, the ridgelet transform still has limitations when dealing with functions with curve singularities. Moreover, in practical images, edges are rarely straight lines, which limits the widespread application of the ridgelet transform. Additionally, the ridgelet transform suffers from high computational redundancy. Therefore, in the last year of the 20th century, Candes and Donoho developed the curvelet transform and formulated its theory. The curvelet transform effectively captures the singular points and characteristics of curves in images. The basis of the curvelet transform exhibits directionality and anisotropy, allowing for the optimal approximation and description of nonlinear parts of 2D images. Thus, curvelet analysis combines the properties of wavelet multiresolution and locality with directionality, providing an optimal approach for representing images with these three features.

2.1.3. Curvelet Transform

The curvelet transform, introduced by E.J. Cande and David L. Donoho at the end of the 20th century, is a method aimed at addressing the limitations of the wavelet transform in expressing the anisotropy of image boundaries and line features [24]. While the wavelet transform is suitable for representing isotropic objects, it falls short in capturing the directional variations in image boundaries and line features. Therefore, the curvelet transform was proposed as a first-generation extension of the wavelet transform, preserving the advantages of multiresolution and time-frequency localization while introducing the ability to perform multiscale, translation, and multidirectional transformations. This enables the curvelet transform to better describe the singular changes in images, which cannot be effectively expressed by the wavelet transform.

Numbered lists can be added as follows:

• First-generation curvelet. The first-generation curvelet transform primarily involves subband decomposition and a multiscale ridgelet transform, which is a nonadaptive representation method. As shown in Figure 1, the first-generation curvelet includes subband decomposition, smoothing partitioning, normalization, and ridgelet analysis. The decomposition process introduces significant data computation, making it quite complex in digital implementation. Based on this, E.J. Candes and Donoho proposed a second-generation transform three years later, and this transform is an improved curvelet algorithm that is easier to understand and simpler to implement. Furthermore, the two scholars introduced a fast curvelet algorithm that directly divides frequencies, eliminating the need for the ridgelet transform. This construction significantly differs from the first-generation curvelet algorithm, resulting in a reduction in data and computational complexity.
• Second-generation curvelet transform. The construction of the second-generation curvelet transform differs from the construction of the first generation transform, but the second-generation transform is an improvement of the first-generation transform. The idea behind the first-generation curvelet transform is to partition the target function space into infinitely small blocks to approximate curves as straight lines; then, local ridgelet analysis is utilized. In contrast, the second-generation curvelet transform only retains the abstract principles of the ridgelet transform, such as the framework and tight frame, while discarding its specific computational methods. The continuous curvelet transform serves as an example.
  The formula for the continuous curvelet transform in the time domain is as follows:

  $$C(i,j,k)=〈f,{\phi }_{i,j,k}〉={\int }_{R^2}f\left(x\right)\overline{{\phi }_{i,j,k}\left(x\right)dx}$$

  (11)

  where

  $${\phi }_{i,j,k}\left(x\right)={\phi }_j\left[R_{\theta }\left(x-x_k^j\right)\right]$$

  (12)

  indicates the position at

  $$x_k^{\left(j\right)}=R_0^{-1}\left(k_1\times 2^{-j},k_2\times 2^{-j/2}\right)$$

  (13)
  When the angle position of all points at 2(${}^{-j}$) is considered, the curvelet transform can be obtained by rotating the curvelet basis ${\phi }_j$ by a certain angle $\theta$ and shifting it by K.

  $$\theta =2\pi \times 2^{-[j/2]}\times l,l=0,1,\dots ,0\le {\theta }_1\le 2\pi$$

  (14)
  The parameter $k=\left(k_1,k_2\right)\in z^2$ represents the displacement sequence in the 2-dimensional space.

2.1.4. Curvelet Transform Algorithm

The algorithm for the curvelet transform is described as follows:

(1)

Perform a J-level decomposition of the original image I to obtain the subband sequences CJ and DJ;

(2)

Set the initial block size (i.e., the finest scale block) to BMIN, and let B1 = BMIN;

(3)

For j = 1 to J:

-

Divide Dj into subblocks of size BJ with overlapping regions;

-

Apply the discrete ridgelet transform to each subblock;

-

If (j mod 2 = 1), set Bj + 1 = 2Bj; otherwise, set Bj + 1 = Bj.

The curvelet transform itself is a redundant transformation. If the number of wavelet decompositions in the first step is J, then the overall redundancy factor of the entire transform is 16J + 1. Since each step of the above decomposition is reversible, it is possible to construct the corresponding inverse transform algorithm to completely reconstruct the original image.

2.1.5. Analysis of Wavelet Transform Denoising

The wavelet transform method solves the correlation values of the spatial image in a set of established anisotropic bases. The method of wavelet transform denoising is as follows: first, the noisy image is decomposed into subbands of different scales, and the wavelet transform coefficients corresponding to each subband are obtained through the Plancherel operation; then, the obtained wavelet transform coefficients are processed by the hard threshold method (abandoning the smaller transform coefficients and retaining the larger transform coefficients) to filter out noise in the image and retain the edge characteristics of the image. The expression of the hard threshold in the wavelet transform denoising is as follows:

$$C_0(i,h)=\left\{\begin{array}{c}C(i,h),\left|C(i,h)\right|\ge T_i\\ 0,\left|C(i,h)\right|<T_i\end{array}\right.$$

(15)

where $C(i,h)$ is the wavelet coefficient at scale i and direction h, $T_i$ is the threshold corresponding to different scales, and $C_0(i,h)$ is the wavelet transform coefficient after hard threshold denoising; the expression for selecting the threshold $T_i$ is as follows:

$$T_i=k_i,\sigma ·{\sigma }_i$$

(16)

where $\sigma$ is the noise standard deviation, ${\sigma }_i$ is the noise standard deviation corresponding to the subband after transformation, and $k_i$ is the adaptive constant corresponding to each subband. The Monte Carlo algorithm with strong adaptability is used to estimate the noise standard deviation in this method.

2.2. Analysis of Mean Filtering Algorithm

The principle of mean filtering is to calculate the mean of the pixels in the current point neighborhood to replace the gray value of the current point, and this is a linear filtering algorithm. It is equivalent to convolving the image with a filtering template of a certain size, filtering out the objectives with poor correlation, and obtaining a smooth star map background. The formula for mean filtering is as follows:

$$g(x,y)=W(i,j)\ast f(x,y)$$

(17)

where $f(x,y)$ is the original image data, $g(x,y)$ is the filtered image data, and $W(i,j)$ is the filtering template, which is represented as follows:

$$W(i,j)=\frac{1}{M\times N}{\left[\begin{array}{cccc}1& 1& \cdots & 1\\ 1& 1& \cdots & 1\\ ⋮& ⋮& \ddots & ⋮\\ 1& 1& \cdots & 1\end{array}\right]}_{M\times N}$$

(18)

2.3. Mean Wavelet Transform Denoising Process

In a space target image, the background information is primarily represented by pixels with low grayscale values, while the edges and details of the target are mainly represented by pixels with higher grayscale values, assuming there is no contamination from stray light. The curvelet transform is known for its multi-scale and multi-directional sensitivity, which enables effective capture of detailed features in images. However, in practical scenarios, space target images usually contain significant amounts of non-uniform stray light, which can vary in intensity. When the grayscale value of the background information is similar to that of the target edges, the curvelet transform may struggle to effectively capture these subtle changes. As a result, it becomes challenging to distinguish the target from the background, leading to blurry outcomes.

To address this issue, we introduce the mean filtering algorithm to enhance contrast and aid in distinguishing the target from the background. Subsequently, we apply the curvelet transform for filtering. Nevertheless, mean filtering has its limitations, such as insufficient smoothing of the grayscale values in the background information and potential blurring of the target edges. These limitations restrict the denoising capability of the current transform. Furthermore, the non-uniformity of stray light in the space target image can result in regions with no stray light or severely weak stray light, where pixel values already contain valid target information. However, the mean filtering averages over these regions, potentially leading to blurring of object detail.

Hence, in this paper, we propose an image processing approach based on weighted averaging. We denote the two source images as X and Y, where the X image is denoised directly using the curvelet transform and the Y image is filtered by the mean prior to the current transform. The final composite image, denoted as Z, is obtained by taking the weighted average of the two images. The mean curvelet denoising formula can be expressed as follows:

$$C(Z,p)=KC(X,p)+(1-k)C(Y,p)$$

(19)

The weighting coefficient, denoted as k, its expression is as follows:

$$k=\mathrm{a}\times \frac{{\sum }_{x=M_x}^{N_x}{\sum }_{y=M_y}^{N_y}\frac{I(x,y)}{2^N}}{\left(N_x-M_x+1\right)\times \left(N_y-M_y+1\right)}$$

(20)

$M_x,M_y,N_x,N_y$ represent the coordinates of a pixel in the image, $I(x,y)$ represents the value of the pixel at coordinates (x, y), N represents the number of gray levels in the image, a represents an empirical value within the range of $a\in [0,1]$.

In regions of the background where strong stray light is present, we amplify the k value to enhance the proportion of the denoised image Y, which is obtained after mean filtering, in the composite image Z. Conversely, in regions with weak stray light in the background, we reduce the k value to enhance the proportion of the denoised image X, obtained through direct curvelet filtering, in the composite image Z. The flowchart of the mean wavelet transform denoising algorithm is shown in Figure 2.

3. Results

This paper verifies the effectiveness of the mean wavelet transform denoising algorithm through two comparison experiments.

3.1. Lena Image of a Hat Part in the Reverse Color Image Experiment

The resolution of the inverse color image of the hat part of the Lena image is a gray image of 512 × 512 with an 8-bit grayscale. We use the mean filtering algorithm, wavelet filtering algorithm and mean wavelet filtering algorithm to remove the white noise in the original image, and the experimental results are shown in Figure 3 and

Table 1. Quality of the lena image of a hat part reverse color image and experiment images obtained via different methods.

Method	Peak Signal to Noise Ratio	Note
Original noisy image	21.13	-
Mean filtering	24.45	large amount of distortion
Wavelet filtering	28.95	small amount of distortion
Mean wavelet filtering	29.55	very little distortion

.

From Figure 3, the mean wavelet transform algorithm restores the slanted lines on the hat more realistically than the other methods.

3.2. Real Star Image Experiment

The real star image resolution is a 2048 × 2048 gray image with a gray level of 12 bits. We use the mean filtering algorithm, wavelet filtering algorithm and mean wavelet filtering algorithm to filter out the white noise in the original image. The experimental results are shown in Figure 4 and

Table 2. Quality of real star image and experimental images obtained via different methods.

Method	Peak Signal to Noise Ratio	Note
Original noisy image	6.20	-
Mean filtering	10.23	small amount of distortion
Wavelet filtering	6.95	large amount of distortion
Mean wavelet filtering	12.94	very little distortion

.

As shown in Figure 4, in the real star image, the mean wavelet transform can not only remove nonuniform noise but also significantly improve the signal-to-noise ratio of the image.

4. Discussion

This paper proposes the mean-curvelet transform denoising algorithm to address the issues of uneven background intensity, complex noise and interference, and diverse target shapes in images of space targets captured by visible light cameras under the constraints of the imaging system and observation environment. The algorithm applies mean filtering and curvelet transform filtering to the noisy space target images separately and then applies the inverse curvelet transform to the filtered results to generate the final image. The experimental results show that this algorithm effectively improves the signal-to-noise ratio of images.

Furthermore, since the fixed threshold is used in the curvelet transform in this paper, we will study the application of the variable threshold curvelet transform to the algorithm in future experiments.

However, since the curvelet transform is mostly implemented on hardware platforms such as X86 and X64, which cannot tolerate long-term operation in space radiation environments, the algorithm proposed in this paper currently does not have the capability of real-time processing in orbit. In future studies, we will select some hardware platforms with radiation resistance indicators, such as DSP or FPGA, and apply the algorithm on these hardware platforms to achieve real-time processing in orbit.

5. Conclusions

Based on our study, and compared with existing methods, our proposed mean curvelet transform method not only has better suppression performance for images of space targets with uneven background intensity, complex noise and clutter but also maintains good target shape diversity and preserves image details and other information, significantly improving the image quality. Therefore, our proposed algorithm has great application value for denoising space target images. In future research, we will further investigate the performance of the algorithm.