EiCSNet: Efficient Iterative Neural Network for Compressed Sensing Reconstruction

Abstract

The rapid growth of sensing data demands compressed sensing (CS) in order to achieve high-density storage and fast data transmission. Deep neural networks (DNNs) have been under intensive development for the reconstruction of high-quality images from compressed data. However, the complicated auxiliary structures of DNN models in pursuit of better recovery performance lead to low computational efficiency and long reconstruction times. Furthermore, it is difficult for conventional neural network designs to reconstruct extra-high-frequency information at a very low sampling rate. In this work, we propose an efficient iterative neural network for CS reconstruction (EiCSNet). An efficient gradient extraction module is designed to replace the complex auxiliary structures in order to train the DNNs more efficiently. An iterative enhancement network is applied to make full use of the limited information available in CS for better iterative recovery. In addition, a frequency-aware weighted loss is further proposed for better image restoration quality. Our proposed compact model, EiCSNet2*1, improved the performance slightly and was nearly seven times faster than its counterparts, which shows that it has a highly efficient network design. Additionally, our complete model, EiCSNet6*1, achieved the best effect at this stage, where the average PSNR was improved by 0.37 dB for all testing sets and sampling rates.

1. Introduction

With the explosive growth of sensor devices, the acquisition, transmission, and processing of information are proliferating, which brings great challenges for data storage and transmission. In Nyquist sampling theory [1], the sampling process must meet the requirement that the lowest sampling frequency (denoted as the Nyquist sampling frequency) cannot be less than twice the bandwidth in the analog signal spectrum. However, under some extreme conditions with limited bandwidth and battery resources, such as in aerospace or underwater applications, high-quality sampling is difficult to achieve. In addition, radiology has high requirements for a shorter sampling time in order to reduce patients’ burden and discomfort as a result of claustrophobia. Therefore, new sampling theories and methods are needed in order to reduce the amount of sampled data.

Compressed sensing (CS) [2] theory has a high potential to break through the limitations of the traditional Nyquist sampling theory. CS theory demonstrates that information can still be reconstructed if it has been sampled significantly below the Nyquist rate [2]. It senses information as measurements through a measurement matrix. It has been theoretically guaranteed that recovery can be possible if the original signal and the measurement matrix satisfy some mathematical conditions [3]. Then, reconstruction algorithms are employed to decrypt the measurements and estimate the important information in the original signal.

In recent years, many algorithms have been proposed to deal with CS reconstruction. These methods can be divided into two categories: traditional and deep learning (DL) methods. Traditional methods usually have theoretical guarantees and simultaneously inherit interpretability. However, they inevitably suffer from a high computational cost dictated by interactive calculations [4]. In contrast, DL methods realize mapping from compressed data to the original signals by training large numbers of parameters in DNNs. However, the performance of some DL methods is mainly improved by a large DNN model size, ignoring the design of traditional algorithms. Moreover, with the addition of complicated auxiliary structures, it is not possible to make full use of graphics processing units (GPUs), and this severely reduces the run speed. In addition, high-frequency information is difficult to recover because the signals are sampled at a very low frequency.

CS is widely applied in magnetic resonance imaging (MRI) [5], hyperspectral imaging [6], wireless communications [7], space flight [8], and so on. However, these fields are still pursuing higher speeds and better reconstruction performance. For example, in the applications of space flight and MRI, there is hope to achieve a lower pressure of information transmission and a shorter sampling time, respectively, by reconstructing signals that are sampled with a lower compression ratio. Furthermore, imaging or acoustics [9] systems with certain real-time pursuits are eager to achieve the shortest reconstruction time.

In order to address the issues of the existing methods mentioned above and to make the application of CS to systems of signal compression and reconstruction more efficient, we propose an efficient iterative neural network for CS reconstruction (EiCSNet) and make the following contributions:

• Unlike in the SOTA DL methods, a fast elementary reconstruction block based on a gradient-extraction module is proposed in order to realize efficient nonlinear mapping in the reconstruction task to speed up the inference without any complicated auxiliary structures added for inference;
• Considering the known sampling matrix and measurements, the iterative enhancement module makes full use of the limited information available in order to achieve better reconstruction performance with the efficient designs employed in the operations of sampling, upsampling, and reshaping, thus greatly improving the run speed;
• For poor reconstruction at a low sampling rate, a frequency-aware weighted loss that is suitable for CS is further proposed in order to pay more attention to the reconstruction of high-frequency information.

The rest of this paper is organized as follows. The related work is introduced and discussed in Section 2. Our proposed method is described in Section 3. The experimental results are provided in Section 4. Finally, the conclusion is drawn in Section 5.

2. Related Work

In this section, a brief review of CS and landmark work is provided. The methods can be divided into traditional and DL methods.

2.1. Compressed Sensing

CS breaks through the limitations of the Shannon–Nyquist sampling theorem [2]. It helps to simultaneously realize the processes of signal sampling and compression. Instead of directly measuring the signal $\mathit{X}$, it sets a non-adaptive linear projection in order to obtain the overall structure of $\mathit{X}$ at a low sampling rate. Suppose that the signal is $\mathit{X}\in {\mathbb{R}}^{N\times 1}$ and the sampling matrix is $\mathbf{\Phi }\in {\mathbb{R}}^{M\times N}$, $M<<N$; then, the measurement process can be expressed as follows:

$$\mathit{Y}=\mathbf{\Phi }\mathit{X},$$

(1)

where $\mathit{Y}\in {\mathbb{R}}^{M\times 1}$ represents the CS measurement (sampled data). This means that the number of observations M is far lower than that of the signal N. The acquisition is carried out in the form of direct compressive sampling with no other form of sophisticated encoding. Therefore, the burden of quality reconstruction falls solely on the receiver side [10], which greatly reduces the acquisition, transmission, and storage of data.

Generally, underdetermined equations are impossible to solve completely. However, a large number of the signals, such as the natural images, can be approximated as sparse in some domains $\mathbf{\Psi }$ [11]. The sampling projection can obtain a more important structure or information of $\mathit{X}$ in the sparse domain $\mathbf{\Psi }$. On this basis, the CS theory makes the recovery of $\mathit{X}$ from the corresponding measurements $\mathit{Y}$ possible [12]. Then, most CS reconstruction problems can be addressed as optimization problems:

$$\underset{\mathit{X}}{min}{∥\mathbf{\Psi }\mathit{X}∥}_p,\mathrm{such}\mathrm{that}\mathit{Y}=\mathbf{\Phi }\mathit{X},$$

(2)

where $\mathbf{\Psi }\mathit{X}$ is the sparse representation in the $\mathbf{\Psi }$ domain. ${\parallel \ast \parallel }_p$ means the p norm of vector ∗. When p is set to 1 or 0, the norm aims to characterize the sparsity of the vector. Therefore, the reconstruction of compressed signal $\mathit{Y}$ can be understood as finding the solution $\widehat{\mathit{X}}$ of $\mathit{X}$ with the maximum probability of generating $\mathit{Y}=\mathbf{\Phi }\mathit{X}$, where $\mathit{Y}$, $\mathbf{\Psi }$, and $\mathbf{\Phi }$ are the known parameters.

2.2. Traditional Method

Various methods, such as convex optimization, greedy algorithms, and total variation, have been proposed to solve the reconstruction problem.

The convex optimization methods usually translate the $L_0$ norm constraint (nonconvex) in Equation (2) into a $L_1$ norm problem (convex), so that the reconstruction of $\widehat{\mathit{X}}$ can be conducted [13]. They can achieve accurate and robust recovering results. However, the convex optimization methods usually suffer from high computational complexity and the “tweaked” requirement to process image signals.

To reduce the computational complexity, greedy algorithms such as Matching Pursuit (MP) [14] and Projected Landweber (PL) [15] have been proposed for CS reconstruction. Orthogonal MP (OMP) [16] and stage-wise OMP [17] take their source from MP algorithms. They generally supplement the results through the iterative residual. Compared with other traditional approaches, MP methods have relatively low computational complexity at the cost of lowering the reconstruction quality [18,19,20,21,22] and have been proposed as an alternative based on PL. They obtain the reconstructed image by successively projecting and thresholding. They not only have lower computational complexity than convex-programming approaches but are also flexible in terms of incorporating other additional optimization criteria [18]. However, the PL methods have to conduct numerous iterations to obtain the final results, and heavy matrix operations need to be executed in each iteration.

For the task of CS image reconstruction, some existing works have established more sophisticated models with Total Variation (TV) regularized constraint being employed to give more expression to image priors [23,24]. Paying more attention to image priors can obtain the overall recovering images more accurately and robustly but slow down the run speed. In addition, a few details may be lost due to the oversmooth reconstruction.

Though traditional methods have been widely used in many practical projects [25,26], a large number of iterations and heavy matrix operations greatly reduce the operation speed.

2.3. Deep Learning Methods

Due to powerful GPU devices and sufficient data resources, DL methods conducted on GPUs have become possible in the field of CS [27,28]. They have achieved excellent recovery performance and orders of magnitude speed improvement. Refs [27] firstly employed Convolution Neural Network (CNN) in CS and realized the non-iterative reconstruction, which also achieved the highest run speed at that time. However, it still consumed much time because of the picture rearrangement and the heavy fully connected network for upsampling. In [28], the Stacked Denoising Autoencoder was applied to capture statistical dependencies between the different elements of certain signals, which improved the signal recovery performance. In [29], ISTA-Net was proposed for CS reconstruction, with the iterative shrinkage-thresholding algorithm.

Compared with the traditional methods, they have greatly improved the recovery performance and run speed. Through the training of large data sets, DL methods realize more flexible nonlinear mapping, as shown in Equation (3),

$$\widehat{\mathit{X}}=f_{\theta }\left(\mathbf{\Psi },\mathit{Y}\right),$$

(3)

where $f_{\theta }(\ast )$ represents the mapping to solve underdetermined equations with learnable parameters $\theta$. In this way, it can be more consistent with the statistical distribution of real image sets. In addition, it has been the focus of attention to efficiently utilize the information obtained from $\mathit{Y}$.

In [4], a hybrid framework was proposed to leverage the local spatial information from CNN and the global context provided by a transformer. However, its run time is similar to the traditional methods, which is hundreds of times that of the other SOTA DL methods. Ref. [30] unfolded the iterative denoising process of the well-known Approximate Message Passing (AMP) algorithm. It integrated deblocking modules to eliminate the blocking artifacts that usually appear in the CS of visual images. Ref. [31] built a trainable deep compressed sensing model by combining Convolution Generative Adversarial Networks and a Variational Autoencoder. In [32], ISTA-Net++ had the adaptability to handle CS problems with different ratios through a single model.

However, the abovementioned DL methods could not make full use of the large-scale parallel computing capability of GPU devices for three reasons. First, it is difficult to train the huge end-to-end DNNs without auxiliary structures, such as [18]. In this way, the heavy full connection between any two successive layers [27] and the auxiliary structures to strengthen nonlinear operations [4,18,30,33] greatly affects the utilization efficiency of the GPU devices. Second, some methods cut the image into blocks for processing [27] or reshape the upsampled matrix, which spends time on matrix splicing and the subsequent deblocking filtering. Finally, the unreasonable design of a network structure to realize the mapping may lead to needing a deeper network to achieve better results at the cost of more reconstruction time. Therefore, new DL models should be developed to improve the reconstruction speed by fully utilizing the GPU resources.

3. Methods

In order to achieve a higher reconstruction speed and keep the recovery performance as effective as possible, an efficient model EiCSNet is proposed by taking the characteristics of the DL methods and CS algorithms into consideration. The overview of EiCSNet, as shown in Figure 1, comprises the following modules:

• The Gradient-Extraction Module (GEM) and Fast Elementary Reconstruction Block (FERB) aim for a hardware-friendly structure without auxiliary branches to speed up the inference of CS reconstruction (Section 3.1);
• The Iterative Enhancement Module (IEM) better combines the elaborate design of traditional algorithms based on greedy iteration and makes full use of intermediate information to strengthen and supplement images to achieve efficient and high-quality reconstruction (Section 3.2);
• The Frequency-Aware Weighted Loss (FAWL) function is proposed to pay particular attention to the reconstruction of high-frequency details (Section 3.3).

3.1. Hardware-Friendly GEM and FERB

Stochastic Gradient Descent [34,35] is widely used to train neural networks. However, the parameters in deeper networks cannot be fully trained because the gradients gradually decrease in backpropagation [36] with the growth of the layer numbers, which is noted as the vanishing gradient problem [37].

To solve the vanishing gradient problem, Residual Network (ResNet) was introduced in 2015 by researchers at Microsoft Research, which uses a technique called skip connections to skip training from a few layers and connect directly to the output. The advantage of adding this type of skip connection is that if any layer hurts the performance of the architecture then it will be skipped by regularization. In this case, it can effectively avoid network collapse by jumping through some layers. Furthermore, the gradients can also be transferred to a deeper network without too much attenuation.

The basic block of ResNet and traditional CNN options are shown in Figure 2. It can be divided into two branches: the Main Branch (MB) for the main nonlinear calculation and the Residual-link Branch (RB). The RB aims to realize a skip-connection from the deeper feature maps to the shallower ones so that the gradient variables can be recorded, preserved, and transferred effectively. When the dimensions of $I/O$ feature maps are changed, there is a need for the RB to go through a $1\times 1$ convolution to keep the residual connection, as shown in Figure 2b.

However, the additional RB places an extra burden on the hardware. The two branches cannot be calculated, completed, and discarded simultaneously, so extra memory space and memory access are required to store and transfer the data. In addition, the RB also needs many operations of point-wise addition, and the additions can only be carried out until the end of the MB. As a result, the different operations of the two branches cannot make full use of the GPU, seriously degrading the parallel performance.

Table 1. The comparison of three basic structures. Each structure is stacked ten times. The total times of the three structures are listed, which are tested for 1000 rounds under various inputs and outputs of dimensions $W\ast H\ast C$.

	Structure	Figure 2a	Figure 2b	Figure 2c
$\mathit{W}\ast \mathit{H}\ast \mathit{C}$
$4\times 4\times 512$		3.32	3.82	2.92
$8\times 8\times 256$		2.92	3.75	2.39
$16\times 16\times 128$		2.40	3.18	1.87
$32\times 32\times 64$		2.36	3.13	1.85
$64\times 64\times 32$		2.42	3.23	1.94
$128\times 128\times 16$		2.35	3.12	1.86

lists the inference time of three basic structures in Figure 2. Under various input and output conditions, the structure in Figure 2a is faster than the variant one in Figure 2b. It can also be clearly seen that the res-free structure (Figure 2c) achieves the best performance in terms of inference time. This phenomenon is more obvious when the input and output feature maps become larger.

To address the vanishing gradient issue while achieving a high reconstruction speed, we propose a hardware-friendly Gradient-Extraction Module (GEM), as shown in Figure 3. The blue cubes and gray boxes in Figure 3 express feature maps and several convolution operations, respectively. The arrows with different sizes and shades of green represent the gradient calculated in the process of backpropagation. The outputs of each stage marked by green boxes are calculated from the intermediate feature maps in the forward process through GEMs. In the main branch, the smaller and lighter arrows represent how the overall size of the gradient gradually decreases or even disappears with the deeper network. It is noted that all CNN blocks work together to minimize the difference between the final output and the ground truth. In this way, the structure of the RB in Figure 2a,b is replaced by the GEM in the proposed reconstruction neural network. The deeper parameters obtain an extra effective gradient to drive faster and more effective training. In addition, it can also play a restrictive role in the training of deep layers to avoid fading away from the goal. The parameters of the deeper layers continuously perceive the objective loss function through the leaf branches of the GEMs. In this way, deeper leaf branches can help to transfer and enhance the gradient to achieve a better complete training effect, even if parts of the gradient disappear. Because the parameters of shallower layers are trained easily, there is no need to add GEM to assist in the training. GEM not only helps to train deep layers of the neural network but also greatly improves the speed of the inference due to the removal of the residual branch.

The GEM was employed as the understructure of the Fast Elementary Reconstruction Block (FERB) to realize fast and efficient reconstruction. The testing and training pipelines of the FERB are shown in Figure 4a,b, respectively. Both input and output channels were set to 64. Relu modules, which only introduced a few hardware overheads, were applied to achieve the nonlinear fitting and the deeper stacking of network modules. The GEM helps to train parameters in deep layers for better results without any auxiliary structures. It should be noted that the GEM was removed to improve the inference speed because no intermediate stage outputs were required to assist the gradient.

3.2. Iterative Enhancement Module

The mapping solution of the DL is accompanied by a large number of parameters to realize the nonlinear transformations from the measurements to the original images. However, they are suspected of producing violent solutions that just fall into universal expressions of the golden mean, which is subject to the overall distribution of the training data. In this way, the final reconstruction tends to the distribution of the training datasets and the recovery of low-frequency information that is easy to be realized or just belongs to the common pattern. To this extent, the network practicability also becomes highly dependent on whether the testing data have a similar distribution as the training data. These methods do not connect and develop the predecessors’ exploration research and will have some weaknesses for practical use. Other DL methods such as [30] are firmly entrenched in the process of traditional methods to carry out the upsampling by ${\mathbf{\Psi }}^T\mathit{Y}$. In this way, the high dimensional nonlinear mapping realized by a large number of parameters in DL can not be fully utilized. To address the above issue, the effective Iterative Enhancement Module (IEM) based on a greedy reconstruction algorithm is proposed, which contains the following advantages:

• Through hardware-friendly structures and high-speed parallel operation, better performance and faster speed can be obtained;
• Better results can be obtained as the number of iterations increases;
• Combined with the characteristics in the field of CS, the network structure can be explained to a certain extent.

The main flow of the IEM is illustrated in Figure 5. In the first iteration $i=1$, the input sampled data ($\mathbf{SD}$) were sampled through the sampling matrix and were reconstructed as the initial reconstruction image ${\mathbf{IRI}}_1$:

$$\begin{array}{cc}\hfill \mathbf{SD}& =\mathbf{X}\ast \mathbf{SC}\hfill \\ \hfill {\mathbf{IRI}}_1& =reshape(\mathbf{SD}\ast {\mathbf{UC}}_1)\hfill \end{array},$$

(4)

where $\mathbf{X}$ and $\mathbf{SD}$ mean the input of the real image and the sampled data, respectively. $\mathbf{SC}$ and ${\mathbf{UC}}_1$ represent the sampling matrix and the upsampling matrix realized by convolution $32\times 32\times 1\times (1024\ast r)$ and $1\times 1\times (1024\ast r)\times \left(1024\right)$ (r represents the sampling ratio), respectively. Furthermore, the $1\times 1$ convolution and pixel shuffle were used to stretch and reshape the upsampled information. The understructure of the upsampling and reshaping module is shown in Figure 6. All operations were easily deployed to the GPU to achieve high efficiency. The comparison of the performance, run time, and the settings of $N_I,N_B$ are provided in Section 4.

In the following iteration $i=2,3,\cdots N_I$, the stage sampled data (${\mathbf{SSD}}_i$) were achieved by sampling ${\mathbf{SO}}_{i-1,N_B}$. ${\mathbf{SO}}_{i-1,N_B}$ represents the stage output after $N_B$ FERB operations (mentioned in Section 3.1) in iteration $i-1$. Thereafter, the remaining error between $\mathbf{SD}$ and ${\mathbf{SSD}}_i$ was calculated. The Error Enhancement Image ${\mathbf{EEI}}_i$ was stretched from the remaining error through upsampling and reshaping from the remaining error in the same way as $i=1$ (defined in Equations (4) and (5)). The ${\mathbf{EEI}}_i$ was added to the last iteration result ${\mathbf{SO}}_{i-1,N_B}$ to strengthen it and compensate for its errors. In this process, the ${\mathbf{IRI}}_i$ was calculated as follows:

$$\begin{array}{cc}\hfill {\mathbf{SSD}}_i& =({\mathbf{SO}}_{i-1}\ast \mathbf{SC})\hfill \\ \hfill {\mathbf{EEI}}_i& =reshape((\mathbf{SD}-{\mathbf{SSD}}_i)\ast {\mathbf{UC}}_i)\hfill \\ \hfill {\mathbf{IRI}}_i& ={\mathbf{SO}}_{i-1,N_B}+{\mathbf{EEI}}_i,\hfill \end{array}$$

(5)

where ${\mathbf{UC}}_i$ represents the upsampling matrix to stretch the error in the compressed domain $(\mathbf{SD}-{\mathbf{SSD}}_i)$. Therefore, we realized various mapping interpretations to supplement the ${\mathbf{IRI}}_i$, which had a greater effect on the recovery process. In this way, the different supplementary information in different iteration rounds was obtained through the sampling error to strengthen the final results.

The stage ${\mathbf{SO}}_{i,j}$ was calculated as follows:

$${\mathbf{SO}}_{i,j}=\left\{\begin{array}{cc}FER{B}_{i,j}\left(CN{N}_{1-64}\left({\mathbf{IRI}}_i\right)\right),\hfill & \mathrm{if}j==1\hfill \\ CN{N}_{64-1}(FER{B}_{i,j}\left(\left({\mathbf{IRI}}_i\right)\right),\hfill & \mathrm{if}j==N_B\hfill \\ FER{B}_{i,j}\left({\mathbf{SO}}_{i,j-1}\right),\hfill & \mathrm{otherwise}.\hfill \end{array}\right.,$$

(6)

Here, $FER{B}_{i,j}(·)$ means the j-th option in iteration i of the FERB. After $N_B$ FERB operations, the final stage output in iteration i can be represented as ${\mathbf{SO}}_{i,N_B}$. To adapt to the I/O channel needs between the IEMs and FERBs, two convolution layers ($CN{N}_{1-64}$ and $CN{N}_{64-1}$) were added to adjust the channel.

The pseudo-code is provided in Algorithm 1. The proposed method realized a reconstruction algorithm stage by stage. $\mathbf{SD}$ was absolutely calculated from the real image. As a result, we could keep employing this information to guide the reconstruction network. In this way, ${\mathbf{EEI}}_i$ was calculated from ${\mathbf{SO}}_{i-1,N_B}$ and the real sampled data $\mathbf{SD}$ at the beginning of each subsequent iteration $i=2,3,\cdots N_I$ (seen in Equation (5)). They were expected to play an important role in the induction and summary of the previous iteration results and were also regarded as important compensation to guide the next response.

Algorithm 1 PREDICTION of IEM

1: PREDICT (Sampled data $\mathbf{SD}$) 2:      for each $i\in [1,N_I]$ do 3:            if $i$ == 1 then 4:                  ${\mathbf{IRI}}_i\leftarrow reshape(\mathbf{SD}\ast {\mathbf{UC}}_1)$ 5:            else 6:                  ${\mathbf{SSD}}_i\leftarrow ({\mathbf{SO}}_{i-1,NB}\ast \mathbf{SC})$ 7:                  ${\mathbf{EEI}}_i\leftarrow reshape((\mathbf{SD}-{\mathbf{SSD}}_i)\ast {\mathbf{UC}}_i)$ 8:                  ${\mathbf{IRI}}_i\leftarrow {\mathbf{SO}}_{i-1,NB}+{\mathbf{EEI}}_i$ 9:            end if 10:         for each $j\in [1,N_B]$ do 11:                if $j==1$ then 12:                      ${\mathbf{SO}}_{i,1}\leftarrow FER{B}_{i,1}\left(CN{N}_{1-64}\left({\mathbf{IRI}}_i\right)\right)$ 13:                elif $j==N_B$ then 14:                      ${\mathbf{SO}}_{i,N_B}\leftarrow CN{N}_{64-1}\left(FER{B}_{i,1}\left({\mathbf{IRI}}_i\right)\right)$ 15:                else 16:                      ${\mathbf{SO}}_{i,j}\leftarrow FER{B}_{i,j}\left({\mathbf{SO}}_{i,j-1}\right)$ 17:                end if 18:           end for 19:      end for 20:      return ${\mathbf{SO}}_{N_I,N_B}$

Our proposed network aims not only to reconstruct the final output to be similar to the target but also to obtain the following detailed goals. On the one hand, in the continuous iterations, the IEMs and FERBs assume that the intermediate stage outputs ${\mathbf{SO}}_{i,N_B}$ can gradually approach the target, reflected by the pursuit of the minimization error between ${\mathbf{SSD}}_i$ and $\mathbf{SD}$ in the compressed domain. In addition, the ${\mathbf{EEI}}_i$ turns darker, indicating that the error between ${\mathbf{SO}}_{i,N_B}$ and the ground truth is gradually reduced to zero in continuous interations. On the other hand, it also means that the difficulty of reconstruction decreases gradually. Compared with the previous work [18], the network parameters need to realize the complete mapping from the sampled data to the target. However, now there is a series of ${\mathbf{EEI}}_i$ that can supplement, improve, and strengthen the final output at the beginning of each round. Therefore, the parameters can be used more efficiently. In the process of the step-by-step iteration, the error supplement gradually attains the results, which means that the network has a certain interpretability rather than a completely violent solution process.

3.3. Frequency-Aware Weighted Loss

There are two problems encountered in the process of solving the underdetermined reconstruction. First, there is only a small amount of effective information contained in the measurements. To a large extent, it can only guide the reconstruction of low-frequency energy. Second, the DL methods are generally updated in the training. Because of the common encounter of low-frequency patterns, the network will gradually find the golden mean method that will not obtain the extreme penalty from the objective function. In such a scenario, it will excessively pursue the overall mean performance to achieve stability and reliability, rather than the integrity of the reconstructed information at fewer high-frequency points.

To address the abovementioned issue, the compensation method Frequency-Aware Weighted Loss (FAWL) is proposed. We expanded the outer edge by symmetric mapping. The examples of the original image and the extended image $\mathbf{EI}$, with the expansion length $E=1$, are shown in Figure 7. The edge mapping means that pixels at the original edge could also be applied to the following formulas. In addition, it also ensured that the calculated weight information at the edge would not be ignored though there were insufficient surrounding pixels.

The frequency-aware coefficient masks $\mathbf{CM}$ in Figure 8 show that the farther away from the target pixel $(i,j)$, the darker the colors that emerged from the $\mathbf{CM}$. In other words, long-distance pixels had less impact on calculating the high-frequency parts of the information.

The $\mathbf{CM}$ was calculated as follows:

$$\begin{array}{cc}\hfill D_{i,j,m,n}& =\sqrt{{\left(i-m\right)}^2+{\left(j-n\right)}^2}\hfill \\ \hfill {\mathbf{CM}}_{i,j,m,n}& =\left\{\begin{array}{cc}1,\hfill & i=m,j=n\hfill \\ \frac{1}{D_{i,j,m,n}},\hfill & 0<\left|i-m\right|,\left|j-n\right|\le E\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\hfill \end{array}$$

(7)

Here, $D_{i,j,m,n}$ means the distance between pixel $(i,j)$ and $(m,n)$. ${\mathbf{CM}}_{i,j,m,n}$ represents the coefficient mask, and only pixels in the range of E are perceived. E can also be understood as the range of frequency perception.

The changing information of the surrounding pixels within the range E was multiplied by the $\mathbf{CM}$. The comprehensive frequency characteristic weight of the central pixel $(i,j)$ was then obtained by synthesizing the frequency gradient. The frequency-aware weighted feature ($\mathbf{FAWF}$) was calculated through (8).

$$\begin{array}{cc}\hfill {\mathbf{FAWF}}_{i,j}& \hfill =\sum _{m=i-E}^{i+E}\sum _{n=j-E}^{j+E}\frac{{\mathbf{CM}}_{i,j,m,n}|E{I}_{i,j}-E{I}_{m,n}|}{{(2E+1)}^2}\hfill \end{array}$$

(8)

Figure 9 shows the origin image and the results of the $\mathbf{FAWF}$ with $E=1,10,$ and 50. Through the comparison of different $\mathbf{FAWF}$s, it can be found that the range of the frequency perception was changed when E was selected differently. When the range E was relatively small, the $\mathbf{FAWF}$ was very sensitive to the rapid pixel change in the edge and better provided a higher concentration for high-frequency information. When E was set relatively large, it acted as a broader frequency sensing effect, paying more attention to the wider scale information. The surrounding information was more displayed in the form of color blocks in $\mathbf{FAWF}$ when $E=10$ and 50. When the setting $E\to \infty$, the $\mathbf{FAWF}$ tended to a constant color area and represented an invalid feature without any key weighted information.

After obtaining the $\mathbf{FAWF}$, the FAWL was calculated as follows:

$$\begin{array}{cc}\hfill MS{E}_F(\mathbf{O},\mathbf{T})& =\lambda \sum _{i=0}^{W-1}\sum _{j=0}^{H-1}\frac{{\left({\mathbf{O}}_{i,j}-{\mathbf{T}}_{i,j}\right)}^2\ast {\mathbf{FAWF}}_{i,j}}{WH}\hfill \\ \hfill FAWL(\mathbf{O},\mathbf{T})& =\frac{MSE(\mathbf{O},\mathbf{T})+MS{E}_F(\mathbf{O},\mathbf{T})}{2},\hfill \end{array}$$

(9)

where $W,H$ represent the size of output image $\mathbf{O}$ and target image $\mathbf{T}$, respectively. $\lambda$ represents the balance between $MSE(\mathbf{O},\mathbf{T})$ and $MSEF(\mathbf{O},\mathbf{T})$, where $\lambda$ was set to 10 to ensure that they were in the same order of magnitude and could be regarded as empirical parameters.

Finally, combined with the IEM, the loss function was set as follows:

$$\begin{array}{c}\hfill loss=\frac{1}{{\sum }_i^{N_I}{\lambda }_i}\sum _i^{N_I}{\lambda }_i[2f\left({\mathbf{SO}}_{i,N_B}\right)+f\left({\mathbf{IRI}}_i\right)+\sum _j^{N_B}f\left({\mathbf{GEMO}}_{i,j}\right)],\end{array}$$

(10)

where ${\lambda }_i$ was set to 10 when $i=N_I$ and 5 otherwise to retain the feedback balance between each iteration and pay more attention to the last one. $f(·)$ represents $FAWL(·,\mathbf{T})$, which was defined in Equation (9). The corresponding output of the GEM was expressed as ${\mathbf{GEMO}}_{i,j}$.

The $FAWL$ can be perfectly integrated with all loss functions based on the $MSE$. It is very suitable for the proposed reconstruction method because different concerns of high-frequency information can be introduced into different GEMs. The only disadvantage is that the early data preprocessing may require more preparation time compared with the direct use of $MSE$. However, the time will not be reflected in the process of inference. In Section 4.7, we evaluate the effectiveness of the loss.

4. Experiment

4.1. Datasets

The datasets for training and testing were prepared according to the experimental details in [18]. The BSDS500 [38] was applied as the training set. The 400 images in BSDS500 were cropped as small patches of 96 × 96 pixels with 57 strides. One patch was augmented into eight (i.e., the original image, flipped, rotation 90, rotation 90 plus flipped, rotation 180, rotation 180 plus flipped, rotation 270, and rotation 270 plus flipped). Finally, all 89,600 images were used as the training set. Set5 [39], Set14 [40], and BSD100 [41] were employed as the testing sets because they have been widely used in almost all similar tasks and works. The specific information of these datasets is listed in

Table 2. Summary of the datasets.

Dataset	Number	Comments
BSDS500	500	400 for training
Set5	5	5 for testing, unfixed size
Set14	14	14 for testing, unfixed size
BSD100	100	100 for testing, fixed size

.

4.2. Index

To quantitatively evaluate the performance of different methods, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) were employed. The PSNR is commonly used to quantify the reconstruction quality of images and videos. The PSNR was derived as:

$$\begin{array}{cc}\hfill \mathrm{MSE}& ={\displaystyle \frac{1}{WH}}\sum _{i=0}^{W-1}\sum _{j=0}^{H-1}{\left[{\mathbf{O}}_{i,j}-{\mathbf{T}}_{i,j}\right]}^2\hfill \\ \hfill \mathrm{PSNR}& =10log\left[{\displaystyle \frac{{\mathrm{MAX}}^2}{\mathrm{MSE}}}\right],\hfill \end{array}$$

(11)

where $\mathrm{MAX}$ represents the range of pixel values and is calculated as 255 in the case of 8-bit images. W and H are the width and height of the images, respectively. In addition, $\mathbf{O}$ and $\mathbf{T}$ are the matrices of the output image and target label image, respectively.

SSIM [42], with the best possible value being 1, pays more attention to capturing the structural features of images. It reflects the similarity of the two images, and a larger SSIM indicates better performance [43]. The $\mathrm{SSIM}$ was derived as:

$$\begin{array}{c}\hfill \mathrm{SSIM}=\frac{\left(2{\mu }_{\mathbf{O}}{\mu }_{\mathbf{T}}+c_1\right)\left(2{\sigma }_{\mathbf{O}\mathbf{T}}+c_2\right)}{\left({\mu }_{\mathbf{O}}^2+{\mu }_{\mathbf{T}}^2+c_1\right)\left({\sigma }_{\mathbf{O}}^2+{\sigma }_{\mathbf{T}}^2+c_2\right)},\end{array}$$

(12)

where ${\mu }_{\ast }$, ${\sigma }_{\ast }$, and ${\sigma }_{\mathbf{O}\mathbf{T}}$ are the mean of matrix ∗, the standard deviation of matrix ∗, and the covariance between matrix $\mathbf{O},\mathbf{T}$, respectively. $c_1$ and $c_2$ are two constants to avoid dividing by 0 and were set as ${(0.01\ast \mathrm{MAX})}^2$ and ${(0.03\ast \mathrm{MAX})}^2$, respectively.

When processing multichannel images, they are converted into $\mathrm{YCbCr}$ format, and then the $\mathrm{PSNR}$ and $\mathrm{SSIM}$ can be calculated on the $\mathrm{Y}$ channel as the result [18].

4.3. Settings

The settings of the training hyperparameters are detailed in

Table 3. Summary of training hyperparameter settings.

Parameters	Value
Batch size B	64
Learning rate for REC $L_{REC}$	0.0001
Learning rate for SAM $L_{SAM}$	0.00001
Epoch E	300

. Due to the IEMs, the sampling matrix in our network structure was used $N_I$ times in the iterative process. The parameters for the reconstruction process were updated with the change in the sampling matrix. Therefore, it is not recommended for the sampling matrix to fluctuate significantly in the iterative process. In this way, the learning rate of the sampling parameters ($L_{SAM}$) was set to be smaller than that of the reconstruction parameters ($L_{REC}$). The construction and the adjustment process of the model were both implemented on the open-source framework Pytorch 1.6.0 with Python v3.7. All experiments were conducted on a CPU (Intel Xeon CPU E5-2678 v3 @ 2.50 GHz) and one GPU (GeForce RTX 1080 Ti).

Four traditional methods, including discrete wavelet transform (DWT) [44], total variation (TV) [23], multi-hypothesis (MH) [21], and sparse group representation (GSR) [45], were used for comparison in terms of the run time and the quantitative evaluation of the PSNR and SSIM. We also implemented four methods based on DL, including ReconNet [27], ISTANet++ [32], CSNET [18], and AMPNet [30], as the comparative baseline under the same software framework and hardware environment. The four DL works were reproduced according to their papers and tested by the unified testing sets introduced in Section 4.1. It should be noted that if the performance of the reproduced module was lower than the results provided in their papers, we took the specific performance index from the original works. In addition, ReconNet [27] needed to be cascaded with a BM3D denoiser. The BM3D took more than ten seconds for each 256 × 256 image in practical use. Therefore, in the testing of the DL methods, the qualitative and quantitative evaluations were both conducted to compare their performance more intuitively, without any auxiliary filter or denoiser cascaded.

There were two hyperparameter settings in our proposed model: the number of iterations $N_I$ and the number of blocks $N_B$ in each iteration, which had a decisive impact on the run time and performance of the whole network. As a representative, two parameter settings were employed to show the performance and applied to different situations. The first was $N_I=2,N_B=1$, and the second was $N_I=6,N_B=1$, which were represented as EiCSNet2*1 and EiCSNet6*1, respectively. EiCSNet2*1 was a smaller and faster choice, which could achieve the highest speed under the condition of excellent reconstruction performance. EiCSNet6*1 could achieve the best recovery performance at this stage, and its run time was also less than other SOTA methods.

4.4. Quantitative Evaluation

Table 4. The performance results from the different methods. All methods were tested with the three testing datasets and the seven sampling ratios. The indicators are shown in $\mathrm{PSNR}/\mathrm{SSIM}$ format. The best indicators are marked in bold for each column.

Method	Ratio	SET5	SET14	BSD100	Method	Ratio	SET5	SET14	BSD100
DWT	0.01	9.27/0.1402	8.97/0.0989	9.63/0.1067	ISPA Net++	0.01	21.47/0.5918	20.69/0.5171	21.62/0.5089
	0.05	14.27/0.3559	14.52/0.2933	14.81/0.2935		0.05	27.24/0.7933	25.41/0.6827	25.17/0.6420
	0.1	24.74/0.7680	24.16/0.6798	23.46/0.6343		0.1	30.71/0.8713	28.14/0.7725	27.15/0.7274
	0.2	30.83/0.8749	28.13/0.7882	27.26/0.7516		0.2	34.43/0.9246	31.30/0.8572	29.74/0.8215
	0.3	33.61/0.9050	30.38/0.8389	29.23/0.8108		0.3	36.77/0.9469	33.61/0.9013	31.72/0.8763
	0.4	35.32/0.9249	31.99/0.8753	30.72/0.8524		0.4	38.62/0.9609	35.52/0.9292	33.49/0.9124
	0.5	36.87/0.9409	33.54/0.9044	32.17/0.8862		0.5	40.32/0.9707	37.19/0.9485	35.18/0.9381
	Avg.	26.42/0.7014	24.53/0.6398	23.90/0.6194		Avg.	32.80/0.8656	30.27/0.8012	29.15/0.7752
TV	0.01	15.53/0.4554	15.26/0.3890	15.98/0.3995	CSNET	0.01	24.18/0.6478	22.83/0.5630	23.76/0.5484
	0.05	23.16/0.6678	22.24/0.5815	23.05/0.5690		0.05	29.74/0.8485	26.93/0.7331	26.78/0.6976
	0.1	27.07/0.7865	25.24/0.6887	25.46/0.6612		0.1	32.59/0.9062	29.13/0.8169	28.53/0.7834
	0.2	30.45/0.8709	28.07/0.7844	27.58/0.7557		0.2	36.05/0.9481	32.15/0.8941	31.05/0.8721
	0.3	32.75/0.9107	30.12/0.8424	29.27/0.8191		0.3	38.25/0.9644	34.34/0.9297	33.08/0.9171
	0.4	34.89/0.9363	32.03/0.8837	30.86/0.8660		0.4	40.11/0.9740	36.16/0.9502	34.91/0.9443
	0.5	36.75/0.9540	33.84/0.9148	32.46/0.9019		0.5	41.79/0.9803	37.89/0.9631	36.68/0.9618
	Avg.	28.66/0.7974	26.69/0.7264	26.38/0.7103		Avg.	34.67/0.8956	31.35/0.8357	30.68/0.8178
MH	0.01	18.08/0.4472	17.23/0.4218	18.21/0.4076	AMP Net	0.01	23.48/0.6103	22.77/0.5502	23.58/0.5367
	0.05	23.67/0.6566	21.64/0.6528	21.36/0.5169		0.05	29.80/0.8443	27.19/0.7336	26.81/0.6973
	0.1	28.57/0.8211	26.38/0.7433	25.16/0.6673		0.1	33.28/0.9096	29.88/0.8247	28.78/0.7861
	0.2	32.08/0.8881	29.47/0.8278	28.09/0.7746		0.2	36.57/0.9466	32.84/0.8960	31.31/0.8714
	0.3	34.06/0.9158	31.37/0.8732	29.85/0.8307		0.3	38.89/0.9633	35.23/0.9364	33.61/0.9186
	0.4	35.65/0.9337	33.03/0.9084	31.35/0.8695		0.4	41.05/0.9732	37.25/0.9521	35.53/0.9453
	0.5	37.21/0.9482	34.52/0.9314	32.86/0.9012		0.5	42.72/0.9818	39.01/0.9648	37.37/0.9627
	Avg.	29.90/0.8015	27.66/0.7655	26.70/0.7097		Avg.	35.11/0.8899	32.02/0.8368	30.99/0.8169
GSR	0.01	18.87/0.4909	17.87/0.4337	18.90/0.4431	EiCSNet 2*1	0.01	24.41/0.6513	23.14/0.5689	23.86/0.5504
	0.05	24.95/0.7270	22.54/0.6140	22.16/0.5682		0.05	30.00/0.8495	27.22/0.7364	26.87/0.7003
	0.1	29.99/0.8654	27.50/0.7705	25.91/0.7071		0.1	33.13/0.9094	29.63/0.8234	28.72/0.7871
	0.2	34.17/0.9257	31.22/0.8642	29.18/0.8156		0.2	36.46/0.9478	32.60/0.8976	31.28/0.8747
	0.3	36.83/0.9492	33.74/0.9071	31.33/0.8723		0.3	38.81/0.9645	34.85/0.9326	33.40/0.9200
	0.4	38.81/0.9626	35.78/0.9336	33.20/0.9096		0.4	40.83/0.9748	36.79/0.9534	35.42/0.9480
	0.5	40.65/0.9724	37.66/0.9522	34.94/0.9359		0.5	42.75/0.9818	38.71/0.9670	37.41/0.9661
	Avg.	32.04/0.8419	29.47/0.7822	27.95/0.7503		Avg.	35.20/0.8970	31.85/0.8399	31.00/0.8209
Recon Net	0.01	20.60/0.5107	20.06/0.4557	21.10/0.4609	EiCSNet 6*1	0.01	24.61/0.6683	23.34/0.5796	23.97/0.5560
	0.05	24.92/0.6608	23.40/0.5768	23.74/0.5587		0.05	30.48/0.8608	27.61/0.7472	27.05/0.7076
	0.1	26.68/0.7294	24.74/0.6380	24.83/0.6147		0.1	33.62/0.9156	30.06/0.8310	28.96/0.7931
	0.2	28.55/0.7944	26.10/0.6988	25.93/0.6693		0.2	37.11/0.9511	33.21/0.9027	31.63/0.8784
	0.3	30.44/0.8465	27.74/0.7603	27.16/0.7262		0.3	39.40/0.9663	35.37/0.9361	33.81/0.9225
	0.4	32.95/0.8985	29.92/0.8347	29.06/0.8061		0.4	41.33/0.9757	37.29/0.9561	35.82/0.9496
	0.5	33.77/0.9094	30.54/0.8519	29.61/0.8255		0.5	43.31/0.9824	39.20/0.9686	37.83/0.9672
	Avg.	28.27/0.7642	26.07/0.6880	25.92/0.6659		Avg.	35.69/0.9029	32.29/0.8459	31.30/0.8249

lists the PSNR and SSIM performance tested on the three testing sets and seven different sampling rates ($1\%,5\%,10\%,20\%,30\%,40\%,$ and $50\%$). The proposed method achieved the best results in all datasets and sampling rates. Compared with the three SOTA methods (ISTANet++ [32], CSNET [18], and AMPNet [30]), our proposed compact model EiCSNet2*1 achieved similar or even slightly better results of $\mathrm{PSNR}$ and $\mathrm{SSIM}$. Furthermore, it had the highest speed (refer to Section 4.6). The results of EiCSNet6*1 had a much better performance improvement, especially under the condition of a low compression rate, such as $r=1\%$. The average $\mathrm{PSNR}$ was increased by nearly 0.4 dB in all datasets and ratios, which is very helpful for the image reconstruction quality and practical use of CS.

Furthermore, some other recent works were taken into consideration. Because these works used different testing sets or the performance in the paper was not successfully reproduced, the indicators of the $\mathrm{PSNR}$ and $\mathrm{SSIM}$ were directly extracted from the paper [46] for comparison, as shown in

Table 5. The performance results of the different methods under the testing datasets SET11 [27] and BSD68 [41]. The indicators are shown in $\mathrm{PSNR}/\mathrm{SSIM}$ format. The best indicators are marked in bold for each column.

Method	Ratio	SET11 [27]	BSD68 [41]
BCS-Net [47]	0.1	29.42/0.8673	27.98/0.8015
	0.3	35.63/0.9495	32.70/0.9301
	0.4	36.68/0.9667	35.14/0.9397
	0.5	39.58/0.9734	36.85/0.9682
	Avg.	35.33/0.9392	33.17/0.9099
OPINE-NET+ [48]	0.1	29.81/0.8904	27.82/0.8045
	0.3	35.79/0.9541	32.35/0.9215
	0.4	37.96/0.9633	34.95/0.9261
	0.5	40.19/0.9800	36.35/0.9660
	Avg.	35.97/0.9469	32.87/0.9045
MADUN [46]	0.1	29.91/0.8986	28.15/0.8229
	0.3	36.94/0.9676	33.35/0.9379
	0.4	39.15/0.9772	35.42/0.9606
	0.5	40.77/0.9832	37.11/0.9730
	Avg.	36.69/0.9567	33.50/0.9236
EiCSNet 2*1	0.1	30.42/0.9177	28.96/0.8517
	0.3	36.46/0.9721	33.69/0.9461
	0.4	38.78/0.9817	35.68/0.9650
	0.5	40.94/0.9879	37.68/0.9773
	Avg.	36.65/0.9648	34.00/0.9350
EiCSNet 6*1	0.1	30.95/0.9240	29.20/0.8560
	0.3	37.19/0.9750	34.09/0.9480
	0.4	39.50/0.9833	36.12/0.9665
	0.5	41.67/0.9886	38.11/0.9782
	Avg.	37.33/0.9677	34.38/0.9372

Note: The indicator values of BCS-Net, OPINE-NET+, and MADUN were extracted from paper [46]. Because the PSNR/SSIM values under the sampling ratios of 0.01, 0.05, and 0.2 were not provided, the other results were compared when the proposed method was trained and tested on the same dataset as in the paper [46].

. The proposed method was trained and tested under the same conditions described in [46] and showed the best performance on each testing set and under every sampling rate.

The better reconstruction performance was due to our proposed method integrating more CS characteristics, the segmentation and simplification of the reconstruction task, and the multilevel enhancement. In this way, the proposed method had better performance.

4.5. Qualitative Evaluation

For the qualitative evaluation, three sampling ratios ($10\%$, $20\%$, and $30\%$) were selected for comparison. Images were randomly selected from each of the three datasets to fully demonstrate the intuitive performance of the reconstruction. To show the details of the results more clearly, we display the full images and enlarged parts simultaneously in Figure 10. In addition, the indicators of the whole images and detailed parts are calculated and listed. Clearer texture information and similar and sharper shapes were shown in the results of the proposed method. Our proposed model had fewer artifacts or blurred parts in the reconstruction images than the conventional counterparts. The comparison between different methods fully showed that our method had greater advantages in the processing of picture details.

4.6. Inference Speed

Images with a fixed size of $256\ast 256$ were employed to compare the run time of the different methods. The batch size B was set as 1 to carry out the actual run time of every single image. All images were cycled 15 times, and the last 10 times were recorded to obtain the average time for each image. The average run time of the different methods is shown in

Table 6. The run time of the different methods.

Methods	Ratio 0.01	Ratio 0.1
DWT	10.3176/-	10.5539/-
TV	2.4006/-	2.7405/-
MH	23.1006/-	19.0405/-
GSR	235.6297/-	230.4755/-
ReconNet	0.5193/0.0244	0.5258/0.0289
ISTA-Net++	0.5550/0.0356	0.5785/0.0377
CSNET	0.8960/0.0262	0.9024/0.0287
AMPNET	0.5440/0.0361	0.5440/0.0396
EiCSNet2*1	0.1737/0.0052	0.1742/0.0054
EiCSNet6*1	0.4970/0.0141	0.4915/0.0141

Note: The run time is shown in the format of CPU(s)/GPU(s). All images were processed with B = 1.

.

From Table 6, it can be seen that the traditional methods had severe shortcomings in speed, even when they were run on the CPU device. The run speed on the GPU of our method was much higher than all existing methods. The EiCSNet2*1 model achieved a similar performance and a nearly seven times speed improvement over the SOTA AMPNet (both shown in Table 4 and Table 6). In addition, our large model EiCSNet6*1 achieved the best PSNR and SSIM performance, ensuring that the speed was still higher than the SOTA methods. The speed improvement benefitted from the hardware-friendly structures, such as the IEM and FERB, which greatly improved the parallel efficiency and reduced the unnecessary memory access. In each iteration process, the completion and enhancement of the image increased the interpretability of the network and simplified the task difficulty of the network. Hence, it was possible to obtain good results with fewer auxiliary operations.

We also compared the detailed processes of sampling, upsampling, and reshaping between all DL methods on the GPU to show the efficiency of the image restoration. From

Table 7. The time consumption of the sampling, upsampling, and reshaping of different methods. All images were processed with $B=1$.

Method		Ratio 0.01	Ratio 0.1	Method		Ratio 0.01	Ratio 0.1
Recon Net	sampling	-	-	AMP Net	sampling	0.00085	0.00094
	upsampling	0.00075	0.00076		upsampling	0.01142	0.01126
	reshaping	0.00330	0.00377		reshaping	0.00619	0.00697
	All	0.00405	0.00453		All	0.01846	0.01917
ISTA- Net++	sampling	0.00392	0.00426	EiCSNet 2*1	sampling	0.00039	0.00036
	upsampling	0.00140	0.00178		upsampling	0.00025	0.00026
	reshaping	0.00511	0.00581		reshaping	0.00018	0.00019
	All	0.01043	0.01185		All	0.00082	0.00081
CS Net	sampling	0.00031	0.00026	EiCSNet 6*1	sampling	0.00096	0.00087
	upsampling	0.00016	0.00016		upsampling	0.00060	0.00066
	reshaping	0.00530	0.00551		reshaping	0.00045	0.00050
	All	0.00577	0.00593		All	0.00201	0.00203

Note: The run time is shown in the format of CPU(s)/GPU(s). All images were processed with B = 1.

and Figure 11, it can be seen that the run time of our method was very stable under different sampling rates. Our method avoided the time loss caused by inefficient networks or auxiliary structures through our hardware-friendly designs.

4.7. Ablation Experiment

In the ablation study, we explored the effectiveness and settings of our proposed network structure to achieve the best performance. Furthermore, we also conducted ablation experiments on the FAWL and GEM to illustrate the improvements they brought to the task of CS recovery.

The Verification of the Module Settings: We calculated the average PSNR and SSIM from the three datasets and seven compression ratios under different network structure settings. They are all detailed in

Table 8. The average PSNR and SSIM for verification of the best model settings. Each PSNR and SSIM is the average of the three datasets and seven ratios.

Settings	PSNR	SSIM	Settings	PSNR	SSIM
$2\times 1$	32.6808	0.8526	$2\times 2$	32.7042	0.8535
$3\times 1$	32.8703	0.8550	$2\times 3$	32.7256	0.8541
$4\times 1$	33.0155	0.8565	$3\times 2$	32.9374	0.8561
$5\times 1$	33.0381	0.8572	$3\times 3$	32.9642	0.8566
$6\times 1$	33.0947	0.8579	$1\times 5$	32.2708	0.8487

and shown in Figure 12 to visualize the differences. The structure settings are identified in $N_I\ast N_B$ format. We set up 10 models ($2\times 1$, $3\times 1$, $4\times 1$, $5\times 1$, $6\times 1$, $2\times 2$, $2\times 3$, $3\times 2$, $3\times 3$, and $1\times 5$) to explore better structure settings.

Because the nonlinear operations were almost concentrated in the block convolution operations, we compared the performance difference of the network structure with the same total number of blocks ($4\times 1$, $2\times 2$, $6\times 1$, $3\times 2$, $2\times 3$, $5\times 1$, and $1\times 5$). It can be seen that, when the total number of convolutions was the same, the higher the iterative reinforcement was, the more effective the network was. Therefore, we found that it was more effective to distribute the convolution calculation more dispersed to each iteration, that is, setting $N_B$ to 1 had better cost performance. Furthermore, it explained the operation of the DL network in making up for the errors in the step-by-step iterative process.

From the data of $2\times 1$, $3\times 1$, $4\times 1$, $5\times 1$, and $6\times 1$, we found that with the increase in the $N_I$, the network performance improved. The experiment ended when $N_I=6$. It was mainly considered that the continuous increase in the network would not greatly improve the performance. Furthermore, we found that EiCSNet6*1 also had more advantages in both speed and performance than the existing methods.

FAWL and GEM: To verify the effectiveness of the functions of each part, we tested the final performance of the two network models at the ratio of 0.01 in four different cases as follows:

• Nothing: neither FAWL nor GEMs were set. Only MSE loss worked;
• $W/O$ FAWL: no FAWL was set, but the GEMs played a part in the training process;
• $W/O$ GEM: no GEM was added, but the FAWL was considered;
• ALL: both FAWL and GEM acted with united strength.

The results of the PSNR and SSIM are tabulated and shown in

Table 9. The average PSNR and SSIM for the ablation experiment. Each PSNR and SSIM was averaged over the three datasets.

Settings	$2\times 1$		$6\times 1$
	PSNR	SSIM	PSNR	SSIM
Nothing	23.65	0.5839	23.79	0.5906
W/O F	23.75	0.5867	23.92	0.5957
W/O GE	23.72	0.5837	23.86	0.5917
ALL	23.80	0.5902	23.97	0.6013

. It can be seen clearly that with the supplementation of FAWL, the network performance improved in different models. Furthermore, with the addition of the GEM, the training of the network became relatively simple and stable to support the better convergence of this task. Additionally, when the network structure tended to be deeper, a larger improvement range was introduced by the GEM. In contrast, in a shallower network structure, there was only a small performance increase.

5. Conclusions

In this paper, EiNet was proposed for better and faster CS image reconstruction. The FERB based on the GEM reduced the additional auxiliary structures and improved the parallel efficiency with no performance degradation. The IEM combined the characteristics and requirements of CS, which made the whole network structure more efficient and compact and obtained a significant performance improvement. FAWL made the image reconstruction network more effective and robust, avoiding the blurred performance in high-frequency information. Our method not only achieved better reconstruction performance but also was nearly seven times faster than other SOTA methods during the inference process. There is a strong potential to run the model on a mobile terminal, which may be valuable to future CS image restoration.