Underwater Image Super-Resolution via Dual-aware Integrated Network

Abstract

Underwater scenes are often affected by issues such as blurred details, color distortion, and low contrast, which are primarily caused by wavelength-dependent light scattering; these factors significantly impact human visual perception. Convolutional neural networks (CNNs) have recently displayed very promising performance in underwater super-resolution (SR). However, the nature of CNN-based methods is local operations, making it difficult to reconstruct rich features. To solve these problems, we present an efficient and lightweight dual-aware integrated network (DAIN) comprising a series of dual-aware enhancement modules (DAEMs) for underwater SR tasks. In particular, DAEMs primarily consist of a multi-scale color correction block (MCCB) and a swin transformer layer (STL). These components work together to incorporate both local and global features, thereby enhancing the quality of image reconstruction. MCCBs can use multiple channels to process the different colors of underwater images to restore the uneven underwater light decay-affected real color and details of the images. The STL captures long-range dependencies and global contextual information, enabling the extraction of neglected features in underwater images. Experimental results demonstrate significant enhancements with a DAIN over conventional SR methods.

1. Introduction

As global pressure on land-based resources continues to grow, increasing attention is beginning to be focused on the oceans and seas as an important source of natural resources. The oceans cover the vast majority of the Earth’s surface, which contains rich biodiversity, energy potential, and mineral resources. However, due to complex factors such as scattering, absorption, and color deviation of light in underwater environments, the quality of underwater scenes is significantly degraded. To address these challenging issues, many methods [1,2,3,4,5,6] have been developed to improve degraded image quality. In the beginning, people tended to make more elaborate filters to recreate as many images as possible captured in underwater environments, largely inspired by the theory of the human retina [7]. Later, physical models were often used to mimic the respective complex environments in real scenarios [8]. In addition to this, they also used a priori methods to globally enhance the picture [9]. Image super-resolution (SR) techniques are designed to restore high-resolution (HR) images from low-resolution (LR) images, which can restore the details and clarity of the images and improve the visualization and application of underwater images. Although the image improvement of these methods performs well in some specific contexts, it may suffer from poor performance when dealing with complex, dynamic, and variable real-world images.

Over the past few years, advances in deep learning have offered fresh solutions for underwater image SR, allowing researchers to better utilize models such as convolutional neural networks (CNNs) to enhance the visual quality and detail of degraded images. Zhang et al. [10] proposed a multipath crossing module that contains both residual and dilation blocks to boost the learning ability of the model and enhance the representation of abstract features. Wang et al. [11] presented a lightweight multi-stage information distillation network, referred to as MSIDN. The MSIDN was designed to strike a better balance between performance and applicability by aggregating locally distilled features from different stages to capture more potent feature representations. Sharma et al. [12] utilized a convolutional block attention module (CBAM)) [13] to specify weights for channel features derived from convolutional neural networks (CNNs) with varying receptive field sizes. Wang et al. [14] proposed a progressive frequency interleaved network called PFIN to enhance and restore underwater images, performing effective color bias correction and detail enhancement in underwater images.

Despite a large amount of previous research on CNN-based methods, previous studies have encountered challenges related to reconstructing optical artificialities (e.g., color distortions) in underwater images. Wang et al. [15] were the first to integrate the HSV color space with deep learning techniques for color correction, noise reduction in the RGB space, and optimization of luminance and satellite images in the HSV space. Li et al. [16] proposed a transmission-guided framework that seeks to enhance feature representation by incorporating multi-color features. Liu et al. [17] proposed an enhancement method based on a super-resolution convolutional neural network (SRCNN) and perceptual fusion by combining learned and unlearned methods, which achieved exciting results in underwater image deblurring and color enhancement. However, existing methods have limitations that do not take into account the global information interaction.

Nowadays, the transformer is widely used in computer vision with its powerful global modeling capabilities. Similarly, various transformer structures are introduced in underwater image restoration tasks to achieve better restoration results. Liu et al. [18] proposed a structure with unique hierarchical segmentation and multiple levels based on a transformer, which overcame the challenge of processing large-size images and effectively captured information regarding long-range dependencies. Peng et al. [19] were the first to apply the U-shaped structure in combination with a transformer to UIE and achieved exciting results. A dual attention transformer-based method was introduced by Shen et al. [20] for underwater image enhancement that can better reconstruct the image. Liang et al. [21] offered the SwinIR method for image restoration using the swin transformer to achieve good image reconstruction. Huang et al. [22] proposed a new adaptive group attention method which was added to the transformer to mitigate the diffusion due to underwater magnesium. Guo et al. [23] used air images to guide underwater image clearness and leveraged the transformer to acquire overall information from the underwater images. Although all of the above methods have yielded some results, they all have certain drawbacks. Models based on CNN approaches may require a large number of parameters and computational resources, and deploying such models on resource-constrained underwater devices may be challenging, which happens to be one of the advantages of transformers. Similarly, transformer-based models are based on a self-attentive mechanism that pays global attention to the entire input sequence. In some underwater scenarios, images may contain a large number of local structures and details, and transformers may not be as effective as methods such as CNNs in dealing with these local structures.

To address these issues, we propose an active dual-aware integrated network (DAIN) for underwater image SR in this paper. The general architecture can be seen in Figure 1, which shows that the DAIN mainly comprises a sequence of dual-aware enhancement modules (DAEMs). A multi-scale color correction block (MCCB) and swin transformer layer (STL) constitute the DAEM, which can utilize the short-range ability of the CNN and the long-range ability of the STL to better integrate local and global information, thereby improving the temporal resolution of degraded underwater images. The MCCB utilizes multiple channels to process the different colors of underwater images to reduce color bias, while the STL captures global contextual information to improve image detail reconstruction. Experimental results display that our DAIN outperforms the most popular methods with low model capacity.

To sum up, our main contributions can be summarized as being three-fold:

• We introduce a lightweight and efficient DAIN tailored for the underwater SR domain. Leveraging the DAME, which consists of an MCCB and STL, our approach demonstrates superior reconstruction performance, outperforming the majority of existing underwater SR methods.
• Our proposed DAEM adeptly integrates local and global features, enhancing the model’s capacity to capture intricate details.
• The MCCB employs multi-channel and multi-scale strategies to address the challenge of uneven color attenuation in underwater images. Notably, the incorporation of the attention mechanism further elevates the representational prowess of the network.

While transformer-based methods excel in global modeling, they exhibit limitations in local modeling. Furthermore, to streamline computations and enhance model efficiency, transformers often entail a significant reduction in computational parameters. Consequently, recent efforts have explored hybrid models that synergize global features extracted by the transformer with local features extracted by a CNN.

2. Related Work

2.1. CNN-Based Underwater Image Super-Resolution

Developing deep learning for various computing tasks shows great benefits [24,25,26,27,28]. Image SR has been a major focus of research in the area of computer vision. It is focused on increasing the quality of photos and videography. SRCNNs [29] first used CNNs in the SR task, which can learn the relationships between paired images, achieving significant improvements over conventional approaches. In order to push the boundaries of reconstruction accuracy, numerous studies have been carried out in this area [30,31,32,33,34,35,36]. In contrast, less research has been performed on underwater images. Owing to the paucity of extensive underwater image datasets, the execution of the aforementioned underwater SR model has some deficiencies. Thankfully, Islam et al. [37] proposed a dataset termed USR-248 that contains paired LR–HR images and devised an underwater SR generative model named SRDRM. To allow for deeper exploration of the underwater SR domain, they also developed an adversarial version of SRDRM, called SRDRM-GAN, which adopts Markov PatchGAN [38] as the discriminator to reconstruct more texture details. Subsequent to this, an increasing number of CNN-based and GAN-based models have been utilized in underwater tasks. Cherian et al. [39] proposed an approach, called the alpha super-resolution generative adversarial network (AlphaSRGAN), to improve restore accuracy. Improving underwater imagery quality results in higher resolution and more concise details. Li et al. [40] proposed WaterGAN for synthesizing underwater images based on indoor images and depth maps. Huo et al. [41] proposed an underwater residual convolutional neural network (URCNN) with great depth based on VGG [42]. Li et al. [43] developed an algorithm for underwater image processing to reduce image blurriness, and they then utilized an end-to-end network to create natural and color-enhanced images. Although these approaches are effective in addressing the undesirable effects of underwater scenes, they lack modeling of global information and they are not conducive to generating more natural and realistic textures.

2.2. Transformer-Based Underwater Images

In contrast to CNN-based methods, Alexey et al. [44] suggested that the direct application of a pure transformer to a sequence of patches in a visual transformer (ViT) could perform the task of image classification well. Liu et al. [18] put forth a shifted windows transformer (swin transformer) to address the difficulty of matching transformers from the verbal domain to the visual domain due to the differences between the two domains. Wang et al. [45] put forth an architecture that includes a novel block of locally enhanced window transformers for stripping, denoising, and deblurring. Recently, transformers have gradually been increasingly utilized in underwater imaging with notable success. For example, Peng et al. [19] developed a U-shaped transformer, integrating a channel-level multi-scale feature fusion transformer block and a spatial-level multiscale feature fusion transformer block to model global features so as to enhance underwater images. Ren et al. [46] presented a dual transformer structure based on a U-shaped structure for super-resolution reconstruction of underwater images and achieved exciting results. Zhang et al. [47] developed the WaterFormer, a two-stage network that combines deep learning and an underwater physical model to tackle the numerous distortions found in underwater images as a result of water’s absorption and scattering properties. Sun et al. [48] developed a model comprising a strengthened LeWin transformer block-based encoder and decoder to enhance color accuracy. The authors of [49] constructed a model consisting of a grey-scale attention and phase transformer block for underwater enhancement. Qi et al. [50] constructed an underwater image enhancement network, known as SGUIENet, utilizing semantic information as high-level guidance to improve the acquisition of locally enhanced features. Although transformers perform well in terms of capturing global dependencies, they are relatively weak in terms of local details. This is a challenge for image SR tasks because local features are critical for understanding details in an image. In this paper, we propose a module, called the DAEM, which efficiently models both global and local information and facilitates high-quality underwater image reconstruction.

3. Methods

3.1. Overall Network Architecture

The proposed DAIN, as illustrated in Figure 1, is composed of three phases. Stage 1 strives to obtain shallow feature information; Stage 2 strives to obtain and merge deeper features; and Stage 3 strives to merge dense features and reconstruct underwater images. The input of our network is an LR image ($X\in R^{H\times W\times 3}$) and its output is an HR image ($S\in R^{\mathrm{r}H\times \mathrm{r}W\times 3}$). The height and width of the image are denoted by $H$ and $W$, respectively. The scale factor is represented by $r$.

In Stage 1, we first use a 3 × 3 convolution to extract shallow information:

$$F_0=H_{SFE}(X)$$

(1)

where $H_{SFE}(·)$ is the 3 × 3 convolution operation and the deviation term of the convolutional layer is omitted for simplicity. $F_0$ is then delivered to Stage 2 as an input to the DAEM, which uses two MCCBs and an STL to model the multi-scale and long-range dependencies of features. More detailed information will be described in Section 3.2 and Section 3.3. Assuming the amount of DAEMs is $D$, the result of the dth DAEM $F_{\mathrm{d}}(1\le d\le D)$ can be expressed as:

$$F_d=H_{DAEM}^d(H_{DAEM}^{d-1}\cdots ((H_{DAEM}^1(F_0))))$$

(2)

where $H_{DAEM}^d(·)$ is the operation of the dth DAEM and $F_d$ denotes the result of the dth DAEM. More importantly, all the results of these DAEMs are combined and transmitted to the dense feature fusion block (DFFB), which aggregates all the hierarchical features to generate more expressive feature representations. To ease the learning difficulties, we add a global residual learning strategy. This specific process is defined as follows:

$$F_{DFFB}=H_{DFFB}[F_0,F_1,F_2,\cdots ,F_D]$$

(3)

where $H_{DFFB}(·)$ denotes the operation of the DFFB, which contains a 3 × 3 convolution and a 1 × 1 convolution. $[F_1,F_2,\cdots ,F_d]$ is the combination of all the characters produced by the DAEM. Finally, Stage 3 functions as the element for recuperation. An upsampling operation is utilized to upgrade the depth detail to the HR image required. We use 3 × 3 convolutional layers and sub-pixel convolution to reconstruct the SR image $S$ as follows:

$$S=H_{UP}(F_{DFFB})$$

(4)

where $H_{UP}(·)$ is the function of upsampling.

Based on the existing image SR, the L1 loss can be integrated into the training of the DAIN, which helps maintain rich texture and local structure [51].

$$L=L_1(G)=E_{X,S}[{‖S-D(X)‖}_1]$$

(5)

Here, ${‖·‖}_1$ represents the L1-norm function. The mapping relationship between LR and HR is implemented by $D:\left\{X\right\}\to S$.

3.2. Dual-Aware Enhancement Module (DAEM)

In the realm of underwater image processing, the correction of color distortions and the recovery of texture details in degraded images necessitate meticulous decomposition. Many existing methods tend to overlook the integration of optical and visual perception, relying solely on simple convolution for high-frequency feature extraction. However, the unique characteristics of underwater images demand separate operations for distinct colors due to uneven color recession. Additionally, the entire image is affected by uniform blurring caused by light scattering and absorption issues, which extend beyond localized regions. Consequently, incorporating long-range dependencies becomes crucial to further enhance reconstruction results.

To address the challenges posed by the diverse complexities of the underwater environment and to elevate image reconstruction performance, we propose the dual-aware enhancement module (DAEM). This module captures dual-aware information, integrating both local and global features to progressively generate natural and realistic textures.

As shown in Figure 1, the DAEM consists of two MCCBs, a 3 × 3 convolutional layer, and an STL. The MCCB mitigates color shifts and distortions in underwater images caused by uneven light attenuation. More details will be shown in Section 3.3. Given the input features $F_{\mathrm{d}-1}$, it first undergoes processing in MCCBs to achieve color bias correction and high-frequency detail recovery, and then fuses the different features using a 3 × 3 convolution. Finally, the STL models long-range dependencies to generate powerful feature representation.

3.3. Multi-Scale Color Correction Block (MCCB)

In underwater images, uneven decay of light propagation leads to low contrast and high color distortion leading to color asymmetry, with the degree of decay varying with wavelength. Conventional underwater processing methods usually adopt a global color compensation strategy and ignore the problem that different color channels are subject to different degrees of attenuation in underwater environments. Consequently, we propose an MCCB that differentially handles three color channels and effectively compensates for different attenuation conditions, avoiding the shortcomings of global processing.

As illustrated in Figure 1, the input features were initially divided into three scales through the use of various convolution sizes dependent on the level of underwater light reduction. Since the attenuation of red light is the most significant, we use a 1 × 1 convolution, while green light and blue light utilize 3 × 3 and 5 × 5 convolution kernels, respectively. This is because different sizes of convolution kernels have different performance in terms of image details and a wide range of features. Let us take the first MCCB as an example, where the procedure could be written as follows:

$$F_{R,1}=\sigma (f_{1\times 1}(F_{\mathrm{d}-1}))$$

(6)

$$F_{G,3}=\sigma (f_{3\times 3}(F_{\mathrm{d}-1}))$$

(7)

$$F_{B,5}=\sigma (f_{5\times 5}(F_{\mathrm{d}-1}))$$

(8)

where $f(·)$ represents the convolution operation, subscripts represent convolution kernels, and $\sigma (·)$ represents the Relu activation function.

Subsequently, we splice and fuse the feature information extracted from each of the three colors:

$$F_{cat}=F_{R,1}\odot F_{G,3}\odot F_{B,5}$$

(9)

where $\odot$ denotes the channel connection and $F_{cat}(·)$ denotes the output after the channel connection. Also, to combine the spatial and channel attentional weights, we use a convolutional block attention module (CBAM) [13]. Finally, we fuse and extract more advanced feature information through the CBAM, which can be defined as follows:

$$F_{MCCB}=H_{CBAM}(F_{cat})$$

(10)

where $H_{CBAM}(·)$ denotes the operation of the CBAM. $F_{MCCB}$ is the output of the MCCB.

The CBAM is an effective and efficient attention module that is widely used in many computer vision tasks. It integrates channel and spatial attention mechanisms to improve the model’s ability to perceive different feature channels and spatial locations. Overall, the use of a CBAM module in underwater image processing can assist the neural network in concentrating on significant features, intensifying the details and contrast of the image and mitigating the lighting problem, thereby enhancing the quality and visualization of underwater images.

3.4. Swin Transformer Layer (STL)

The swin transformer originates from the design of the initial transformer layer, which relies on regular multi-head self-attention to model long-range dependence and thus enhance network representation.

As illustrated in Figure 1, with an initial input of $F_{MCCB}\in {ℝ}^{H\times W\times C}$, the swin transformer reshapes the input $\frac{HW}{M^2}\times M^2\times C$ feature first, where $\frac{HW}{M^2}$ represents the total number of windows. Then it performs self-attention calculations simultaneously $h$ times, where $h$ represents the quantity of self-attention heads. For a local window feature $F_{in}^{swt}\in R^{M^2\times C}$, the query, key, and value matrices $Q$, $K$, and $V\in R^{M^2\times C}$ are expressed as

$$Q=F_{in}^{swt}{W}_Q,K=F_{in}^{swt}{W}_K,V=F_{in}^{swt}{W}_V$$

(11)

where $d=\frac{C}{H}$,$W_Q$, $W_K$, and $W_V$ are shared learnable projections over different windows. The attention matrix $Attn(Q,K,V)$ is then computed via the self-attention in the local window.

$$Attn(Q,K,V)=SoftMax(\frac{Q{K}^T}{\sqrt{d}}+b)V$$

(12)

Here, $b$ is the trainable relative positional coding. Multi-head self-attention (MSA) is combined to maintain consistent embedding dimensions. After the attention operation, there are two Multilayer Perceptrons (MLPs) with GELU activation in them. The LayerNorms (LNs) are then added ahead of MSA and the MLP and use residual connections. The transformer’s overall functionality can be summarized as follows:

$$\begin{array}{l}{F}_{\mathrm{int}er}^{swt}=H_{MSA}(H_{LN}(F_{in}^{swt}))+F_{in}^{swt}\\ F_{out}^{swt}=H_{MLP}(H_{LN}(F_{\mathrm{int}er}))+F_{\mathrm{int}er}^{swt}\end{array}$$

(13)

where $H_{LN}\left(\cdot \right)$ denotes the LN operation, $H_{MSA}\left(\cdot \right)$ represents MSA operation, and $H_{MLP}\left(\cdot \right)$ denotes the operation function. On the one hand, the STL exploits regular and shift window partition alternately, realizing efficient information transmission and interaction between different windows. On the other hand, it uses local attention and a shifted window mechanism to greatly reduce computational effort while maintaining strong modeling power. Therefore, we introduce the STL to better catch local features and details, thus enhancing underwater image processing.

4. Experiments

4.1. Experimental Setup

Two open datasets were adopted to train our proposed work, namely USR-248 [52] and UFO-120 [53], as well as the EUVP dataset [52]. USR-248 was the first dataset made for the SR reconstruction task of underwater optical images, which uses 1060 pairs of underwater images for training and 248 pairs of samples for testing. UFO-120 contains over 1500 pairs of training samples and 120 pairs of testing samples. It is important to note that the LR samples in the USR-248 and UFO-120 datasets are acquired by artificial distortion. Both of them follow the standard procedures [54,55] for optical/spatial image degradation and use manually labeled saliency mappings to create pairs of data.

Adam optimization is utilized to minimize the objective function, with the parameters configured as follows: ${\beta }_1=0.9$,${\beta }_2=0.999$, and $\epsilon ={10}^{-8}$. The learning rate is initially set to ${10}^{-4}$ and then decreased by 50% after 300 iterations. In the STL, the number of multi-heads is set to $h$ = 5 and the number of channels is set to $C$ = 50. The performance of image super-resolution (SR) was thoroughly assessed using three criteria: peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM), and the underwater image quality measure (UIQM). Our model is implemented using the PyTorch framework and was executed on an NVIDIA RTX 3080 GPU.

4.2. Experimental Evaluation on the USR-248 Dataset

The assessment of the USR-248 dataset is presented in

Table 1. Quantitative results of different methods in the USR-248 dataset with scale factors of ×2, ×4, and ×8. Bold is the best performance.

Scale	Method	FLOPs (G)	Params (M)	PSNR (dB)	SSIM	UIQM
×2	SRCNN [29]	21.30	0.06	26.81	0.76	2.74
	VDSR [31]	205.28	0.67	28.98	0.79	2.57
	DSRCNN [56]	54.22	1.11	27.14	0.77	2.71
	EDSRGAN [57]	273.34	1.38	27.12	0.77	2.67
	SRGAN [58]	377.76	5.95	28.08	0.78	2.74
	ESRGAN [59]	4274.68	16.70	26.66	0.75	2.70
	SRDRM [37]	203.91	0.83	28.36	0.80	2.78
	SRDRM-GAN [37]	289.38	11.31	28.55	0.81	2.77
	LatticeNet [60]	56.84	0.76	29.47	0.80	2.65
	Deep WaveNet [12]	21.47	0.28	29.09	0.80	2.73
	AMPCNet [10]	—	1.15	29.54	0.80	2.77
	RDLN [62]	74.86	0.84	29.96	0.83	2.68
	DAIN(ours)	85.55	1.16	29.98	0.84	2.77
×4	SRCNN [29]	21.30	0.06	23.38	0.67	2.38
	VDSR [31]	205.28	0.67	25.70	0.68	2.44
	DSRCNN [56]	15.77	1.11	23.61	0.67	2.36
	EDSRGAN [57]	206.42	1.97	21.65	0.65	2.40
	SRGAN [58]	529.86	5.95	24.76	0.69	2.42
	ESRGAN [59]	1504.09	16.70	23.79	0.65	2.38
	SRDRM [37]	291.73	1.90	24.64	0.68	2.46
	SRDRM-GAN [37]	377.20	12.38	24.62	0.69	2.48
	LatticeNet [60]	14.61	0.78	26.06	0.65	2.43
	Deep WaveNet [12]	5.59	0.29	25.20	0.68	2.54
	AMPCNet [10]	—	1.17	25.90	0.68	2.58
	RDLN [62]	29.56	0.84	26.16	0.66	2.38
	DAIN (ours)	21.78	1.18	26.23	0.70	2.56
×8	SRCNN [29]	21.30	0.06	19.97	0.57	2.01
	VDSR [31]	205.28	0.67	23.58	0.63	2.17
	DSRCNN [56]	6.15	1.11	20.14	0.56	2.04
	EDSRGAN [57]	189.69	2.56	19.87	0.58	2.12
	SRGAN [58]	567.88	5.95	20.14	0.60	2.10
	ESRGAN [59]	811.44	16.70	19.75	0.58	2.05
	SRDRM [37]	313.68	2.97	21.20	0.60	2.18
	SRDRM-GAN [37]	399.15	13.45	20.25	0.61	2.17
	LatticeNet [60]	4.05	0.86	23.88	0.54	2.21
	Deep WaveNet [12]	1.62	0.34	23.25	0.62	2.21
	AMPCNet [10]	—	1.25	23.83	0.62	2.25
	RDLN [62]	18.23	0.84	23.91	0.54	2.18
	DAIN (ours)	5.99	1.26	23.97	0.64	2.20

. The proposed DAIN is compared to some popular SR networks, namely SRCNN [29], VDSR [31], DSRCNN [56], EDSRGAN [57], SRGAN [58], ESRGAN [59], SRDRM [37], SRDRM-GAN [37], LatticeNet [60], Deep WaveNet [12], ESRGCNN [61], AMPCNet [10], and RDLN [62]. We can observe that our proposed DAIN attains comparable results when compared to popular methods. In comparison to mainstream SR methods, our method obtains up to 0.08 dB and 0.1 increases in PSNR and SSIM. Although the DAIN fails to obtain optimal values in terms of UIQM, it still achieves positive results. For example, in the case of a scale factor of ×8, compared to AMPCNet, our DAIN lags behind by 0.05 in terms of UIQM but boosts PSNR and SSIM by 0.14 dB and 0.02, respectively. Visual comparisons of the USR-248 dataset are shown in Figure 2. It is evident that our DAIN produces clearer and more natural image reconstruction. SRDRM and SRDRM-GAN generate visible reconstruction artifacts.

4.3. Experimental Evaluation on the UFO-120 Dataset

In the case of

Table 2. Quantitative results of different methods in the UFO-120 dataset with scale factors of ×2, ×3, and ×4. Bold is the best performance.

Method	Params (M)			PSNR (dB)			SSIM			UIQM
	×2	×3	×4	×2	×3	×4	×2	×3	×4	×2	×3	×4
SRCNN [29]	0.06	0.06	0.06	24.75	22.22	19.05	0.72	0.65	0.56	2.39	2.24	2.02
SRGAN [58]	5.95	5.95	5.95	26.11	23.87	21.08	0.75	0.70	0.58	2.44	2.39	2.56
SRDRM [37]	0.83	-	1.90	24.62	-	23.15	0.72	-	0.67	2.59	-	2.57
SRDRM-GAN [37]	11.31	-	12.38	24.61	-	23.26	0.72	-	0.67	2.59	-	2.55
Deep WaveNet [12]	0.28	0.28	0.29	25.71	25.23	25.08	0.77	0.76	0.74	2.99	2.96	2.97
AMPCNet [10]	1.15	1.16	1.17	25.24	25.73	24.70	0.71	0.70	0.70	2.93	2.85	2.88
ESRGCNN [59]	1.53	1.53	1.53	25.82	26.19	25.20	0.72	0.71	0.70	2.98	2.96	2.85
LatticeNet [60]	0.76	0.77	0.78	25.86	26.13	25.10	0.71	0.71	0.70	2.97	2.94	2.94
HNCT [63]	0.36	0.36	0.36	25.73	25.86	24.91	0.71	0.71	0.69	2.96	2.88	2.84
URSCT [46]	11.37	-	16.07	25.96	-	23.59	0.80	-	0.66	-	-	-
RDLN [62]	0.84	0.84	0.84	25.96	26.55	25.37	0.76	0.74	0.73	2.98	2.98	2.94
DAIN(ours)	1.16	1.17	1.18	26.38	26.62	25.56	0.76	0.76	0.72	2.97	2.94	2.92

, our proposed DAIN demonstrates competitive performance, particularly achieving the highest values in terms of PSNR. Our DAIN is slightly inferior to Deep WaveNet in terms of SSIM and UIQM, but the gap is at most 0.05. The quantitative results of AMPCNet lag far behind our method. Compared with transformer-based methods such as URSCT and RDLN, our method yields a 0.42 dB improvement in PSNR at the scale factor of ×2. In the case of ×4, our DAIN has better performance, obtaining a 0.06 increase in SSIM compared to URSCT. Additionally, the DAIN does not surpass the RDLN in terms of UIQM, but the margin is 0.02 at a scale factor of ×4.

The visual comparisons of the UFO-120 dataset are presented in Figure 3. Our proposed DAIN consistently delivers superior results and effectively recovers finer texture features, bringing it closest to the HR image. Notably, LatticeNet exhibits over-saturation, while AMPCNET and Deep WaveNet result in significant blurring artifacts and distortions. In contrast, our proposed method is better equipped to mitigate these adverse effects. This is attributed to the fact that the DAIN comprises a series of chained DAEMs, which effectively capture both local and global image information, thereby enhancing the quality of image reconstruction.

4.4. Experimental Evaluation on the EUVP Dataset

To further show the robustness and effectiveness of the proposed DAIN, we performed enhancement experiments on the EUVP dataset and compared our model with models including UGAN [54], UGAN-P [54], Funie-GAN [52], Funie-GAN-UP [52], Deep SESR [53], and Deep WaveNet [12]. EUVP consists of 11,435 pairs of underwater images for training and 515 pairs of samples for testing. They follow the standard procedures [54,55] for optical/spatial image degradation and use manually labeled saliency mappings to create pairs of data.

Table 3. Quantitative results of different methods in the EVUP dataset. Bold is the best performance.

Numbers	PSNR(dB)	SSIM	UIQM
UGAN [54]	26.45	0.79	2.87
UGAN-P [54]	26.44	0.79	2.91
Funie-GAN [52]	26.16	0.78	2.95
Funie-GAN-UP [52]	25.16	0.78	2.91
Deep SESR [53]	27.03	0.80	3.06
Deep WaveNet [12]	28.56	0.83	3.02
DAIN(ours)	29.13	0.85	3.03

displays the competitive performance of the DAIN proposed by us, attaining the highest values in terms of both PSNR and SSIM. Specifically, PSNR and SSIM are improved by at least 0.57 dB and 0.02, respectively. The DAIN is found to be slightly lower than Deep SESR in terms of UIQM, but the difference is only 0.03. The quantitative results of Funie-GAN-UP are far below our method. A visual comparison of the EUVP dataset is shown in Figure 4. Our DAIN produces much sharper image reconstruction. Most algorithms, including UGAN, Funie-GAN, and Deep SESR, ignore global dependency modeling, resulting in poor performance. By contrast, our DAIN adequately incorporates global dependency modeling whilst also considering local information, enabling our network to acquire more valuable information and enhance image quality.

4.5. Model Analysis

Analysis of varying numbers of DAEMs. To explore the influence of model depth on reconstruction accuracy, we set the number of DAEMs to $D$ = 10, 12, and 14. From

Table 4. Effect of the number of DAEMs on the USR-248 dataset at a scale factor of ×4.

Numbers	FLOPs (G)	PSNR (dB)	SSIM	UIQM
10	18.53	29.83	0.8177	2.7585
12	21.79	29.86	0.8196	2.7724
14	25.04	29.81	0.8177	2.7803

, it can be seen that there is no significant improvement in the reconstruction results as depth increases. This is because the deeper the depth of the network, the more prone network weights are to deactivation, which makes it difficult to improve performance. Considering the size of the network and restoration accuracy, we opted for $D$ = 12 as the number of DAEMs.

Analysis of edge detection. To further investigate different methods for preserving edge features in the restored images, we employed the Canny algorithm [64] to illustrate the benefits of our proposed method. As depicted in Figure 5 and Figure 6, our DAIN exhibits more prominent edge detection features compared to the majority of methods. These outcomes demonstrate the efficiency of our DAIN in extracting feature information and aiding in the recovery of additional texture details, resulting in visually pleasing effects.

4.6. Ablation Study

To clearly show how our proposed components can improve reconstruction performance, we conducted various experiments, as documented in Table 4. We conducted a stepwise removal of the MCCB, the STL, and the cascades within the MCCB to illustrate the effectiveness of each component. Notably, we modified the MCCB, where we changed the different convolutional sizes to a uniform use of 3 × 3 convolution for feature extraction, and named it as a DAIN with a PCB. We retrained the network and obtained another four models.

As is evident in

Table 5. Ablation studies of different components on the UFO-120 dataset at a scale factor of ×4.

Models	Params (M)	PSNR (dB)	SSIM	UIQM
DAIN w/o MCCB	0.93	25.35	0.7132	2.91
DAIN with PCB	1.54	25.34	0.7143	2.91
DAIN w/o STL	0.67	25.26	0.6974	2.89
DAIN w/o DFFB	1.02	24.79	0.6961	2.89
DAIN	1.16	25.41	0.7161	2.92

, when the MCCB is erased, reconstruction performance drops drastically, with PSNR and SSIM dropping by at least 0.06 dB and 0.0029. It can be observed that both the DAIN without an MCCB and the DAIN with a PCB have worse performance than the DAIN alone. Similarly, although the number of parameters decreases after removing the STL and concatenation, model performance decreases substantially, with PSNR and SSIM decreasing by at least 0.15 dB and 0.0187, respectively. In summary, our proposed MCCB, STL, and concatenation components have a beneficial effect on both the recovery performance and computational efficiency of the network.

5. Conclusions

Underwater images display unique characteristics such as light absorption, scattering, and color attenuation, unlike terrestrial images. To bridge this gap, our study introduces an effective approach called the DAIN for SR reconstruction of underwater imagery. Specifically, the DAIN encompasses a series of DAEMs that tackle image distortion and blurriness by exploring both local and global features. MCCBs examine various colors in an underwater image using multiple channels to reduce color bias and restore texture details. Meanwhile, the STL is capable of modeling global features to enhance high-quality images even further. Thanks to the incorporation of an MCCB and an STL in the DAEM, the network can considerably improve the accuracy of underwater SR. It is proven that our model has higher robustness and efficiency. Through experimentation on benchmark datasets, findings demonstrate that the proposed DAIN achieves competitive performance while requiring fewer parameters compared to other popular methods.