High-Speed Wavelet Image Processing Using the Winograd Method with Downsampling

Abstract

Wavelets are actively used to solve a wide range of image processing problems in various fields of science and technology. Modern image processing systems cannot keep up with the rapid growth in digital visual information. Various approaches are used to reduce the computational complexity and increase computational speeds. The Winograd method (WM) is one of the most promising. However, this method is used to obtain sequential values. Its use for wavelet image processing requires expanding the calculation methodology to cases of downsampling. This paper proposes a new approach to reduce the computational complexity of wavelet image processing based on the WM with decimation. Calculations have been carried out and formulas have been derived that implement digital filtering using the WM with downsampling. The derived formulas can be used for 1D filtering with an arbitrary downsampling stride. Hardware modeling of wavelet image filtering on an FPGA showed that the WM reduces the computational time by up to 66%, with increases in the hardware costs and power consumption of 95% and 344%, respectively, compared to the direct method. A promising direction for further research is the implementation of the developed approach on ASIC and the use of modular computing for more efficient parallelization of calculations and an even greater increase in the device speed.

1. Introduction

Wavelets are actively used to solve a wide range of image processing problems in various fields of science and technology, e.g., image denoising [1], reconstruction [2], analysis [3], and video analysis and processing [4]. Wavelet processing methods are based on the discrete wavelet transform using 1D digital filtering. Digital filtering is performed through repeated additions and multiplications and has a high computational complexity. Modern image processing systems cannot keep up with the rapid growth in the volume of digital visual information that needs to be processed, stored, and transmitted. Improving microelectronic devices is one of the state-of-the-art approaches to increase computational efficiency [5]. Many approaches reduce the computational complexity of wavelet image processing, including various hardware architectures [6]. The authors of [7] developed a multidimensional wavelet construction method that constructs multidimensional inseparable wavelet filter banks from two 1D low-pass filters, one of which is an interpolation filter, to improve image processing speed. In paper [8], the authors developed a new 2D transform called the asymmetric 2D Haar transform and extended it to wavelet packets with an exponentially larger number of bases. The authors of [9] developed an algorithm for the 2D discrete wavelet transform of high-resolution images on Internet of Things nodes. All of these methods are based on pixel-by-pixel image processing. The Winograd method (WM) is based on matrix multiplication and is used as a modern alternative to the classic direct method (DM). The WM reduces the computational complexity of image processing due to the simultaneous calculation of several output values, in contrast to the above methods. The processed image is assembled not from a collection of individual pixels, but from fragments of a certain size. This approach reduces the number of computationally complex multiplications by increasing the number of additions. The WM is used to increase the speed of neural network image processing algorithms [10]. In [11], digital filtering algorithms based on the WM were proposed for convolutional layers of neural networks (NNs), which are superior to the fast Fourier transform in terms of the performance of deep NNs when processing large arrays of visual data. Based on this work, architectures [12] and hardware accelerators [13] have been developed to implement WM-based NN image processing algorithms. In [14], a digital filter architecture based on the WM in a residue number system was developed, which accelerated image processing while increasing hardware costs. However, the WM is designed to obtain groups of adjacent values, while wavelet filtering reduces the sampling rate of the digital signal. In this regard, there is a need to generalize the WM to cases of signal downsampling with a stride of two to implement wavelet filtering of images.

The purpose of this paper is to generalize the Winograd method to the case of convolution with downsampling and to increase the computational speed of wavelet image processing using the WM with decimation.

The rest of the paper is organized as follows. Section 2 describes approaches to wavelet image processing based on the DM and the WM. Section 3 presents a high-speed implementation of the discrete wavelet transform using the proposed approach. Section 4 contains discussion of the received results. Section 5 contains the conclusions.

2. Approaches to Wavelet Image Processing on Modern Hardware Architectures

A. Wavelet filtering using the direct method

Wavelet filtering with decimation of a 2D image along DM lines can be represented as:

$$Z\left(u, y\right)={\sum }_{i=1}^{r}N\left(u, 2y+1-i\right)R\left(i\right),$$

where $Z$ and $N$ are the processed and original 2D images, respectively, and $u$ and $y$ are the row and column numbers of pixels processed by the wavelet filter $R$ of order $r$. Wavelet image processing using the DM is performed through two computational channels corresponding to low-frequency and high-frequency wavelet filters. The scheme of 1D wavelet filtering of a DM image fragment is shown in Figure 1, where $R_L$ and $R_H$ are low-pass and high-pass filters and $Z_L\left(u,y\right)$ and $Z_H\left(u,y\right)$ are the processed images containing low- and high-frequency information about the original image, respectively.

All pixels are processed by a pair of wavelet filters of order $r$ and require $2r$ multiplications and $2\left(r-1\right)$ additions in the DM of wavelet processing. Multiplications have a higher computational complexity than additions and require significant resource costs when digital filtering is implemented on modern microelectronic devices. The WM is one of the main alternatives to the classic DM and is discussed below.

B. Digital filtering using the Winograd method

The WM reduces the computational complexity of image processing by simultaneously computing multiple output values through matrix calculations. The WM formula for 1D image filtering has the following general form [15]:

$$Z=A^T\left(\left(GR\right)\odot \left(B^{T}N\right)\right),$$

(1)

where $Z$ is the fragment of the processed image of size $z\times 1$; $R$ is the wavelet filter mask of size $r\times 1$; $N$ is the fragment of the original image of size $n\times 1$, where $n=z+r-1$; $A^T$, $G$, $B^T$ are the transformation matrices of sizes $z\times n$, $n\times r$, $n\times n$, respectively; and $\odot$ is the operator of element-wise matrix multiplication. Algorithms for compiling transformation matrices are described in detail in paper [16]. The notation WM $F\left(n,r\right)$ contains processed image fragments of size $n$ and the order $r$ of the wavelet used. The sizes of the transformation matrices and the original image fragments depend on them. For example, WM $F\left(4,2\right)$ uses matrices

$$A^T=\left(\begin{array}{ccccc}1& 1& 1& 1& 0\\ 0& 1& -1& 2& 0\\ 0& 1& 1& 4& 0\\ 0& 1& -1& 8& 1\end{array}\right), G=\left(\begin{array}{cc}\frac{1}{2}& 0\\ \frac{1}{2}& \frac{1}{2}\\ \frac{1}{6}& -\frac{1}{6}\\ \frac{1}{6}& \frac{1}{3}\\ 0& 1\end{array}\right)B^T=\left(\begin{array}{ccccc}2& -1& -2& 1& 0\\ 0& 2& 1& -1& 0\\ 0& -2& 3& -1& 0\\ 0& -1& 0& 1& 0\\ 0& 2& -1& -2& 1\end{array}\right),$$

constructed at points $0,\pm 1,2,\infty$.

However, the WM in its classic representation is a method based on processing groups of consecutive pixels, while image filtering through two computational channels during wavelet processing is performed with decimation to reduce the signal sampling frequency and reduce computational redundancy. Thus, there is a need to generalize and expand the WM to a more general case, in which a fragment of the processed image may consist of non-consecutive pixels.

C. Filtration using the Winograd method with downsampling

We expand the WM to the case of filtering with decimation for use in wavelet image processing. Let $x_1,x_2,\dots ,x_n,\dots ,x_{n+r-1}$ be the pixel brightness values of a certain fragment of a line of the original image $N$, and $r_1,r_2,\dots ,r_r$ be the coefficients of the wavelet filter $R$. Then, the calculation of the values $z_1,z_2,\dots ,z_n$ of the fragment of the processed image $Z$ can be represented in matrix form:

$$\left(\begin{array}{c}{z}_1\\ z_2\\ \dots \\ z_n\end{array}\right)=\left(\begin{array}{cccc}{x}_1& x_2& \dots & x_r\\ x_2& x_3& \dots & x_{r+1}\\ \dots & \dots & \dots & \dots \\ x_n& x_{n+1}& \dots & x_{n+r-1}\end{array}\right)\cdot \left(\begin{array}{c}{r}_1\\ r_2\\ \dots \\ r_r\end{array}\right).$$

Expand of processing a fragment with 10 pixels $\left(n+r-1=10\right)$ with a fifth order filter $\left(r=5\right)$ to obtain six values $\left(n=6\right)$ of a fragment of the processed image $Z$ at the output:

$$\left(\begin{array}{c}{z}_1\\ z_2\\ z_3\\ z_4\\ z_5\\ z_6\end{array}\right)=\left(\begin{array}{ccccc}{x}_1& x_2& x_3& x_4& x_5\\ x_2& x_3& x_4& x_5& x_6\\ x_3& x_4& x_5& x_6& x_7\\ x_4& x_5& x_6& x_7& x_8\\ x_5& x_6& x_7& x_8& x_9\\ x_6& x_7& x_8& x_9& x_{10}\end{array}\right)\cdot \left(\begin{array}{c}{r}_1\\ r_2\\ r_3\\ r_4\\ r_5\end{array}\right).$$

When wavelet filtering with decimation, the values $z_1,z_3,z_5$ are calculated:

$$\begin{gathered} \left(\begin{array}{c}{z}_1\\ z_3\\ z_5\end{array}\right)=\left(\begin{array}{ccccc}{x}_1& x_2& x_3& x_4& x_5\\ x_3& x_4& x_5& x_6& x_7\\ x_5& x_6& x_7& x_8& x_9\end{array}\right)\cdot \left(\begin{array}{c}{r}_1\\ r_2\\ r_3\\ r_4\\ r_5\end{array}\right)=\left(\begin{array}{c}{x}_1{r}_1+x_2{r}_2+x_3{r}_3+x_4{r}_4+x_5{r}_5\\ x_3{r}_1+x_4{r}_2+x_5{r}_3+x_6{r}_4+x_7{r}_5\\ x_5{r}_1+x_6{r}_2+x_7{r}_3+x_8{r}_4+x_9{r}_5\end{array}\right)= \\ \left(\begin{array}{c}{x}_1{r}_1+x_3{r}_3+x_5{r}_5\\ x_3{r}_1+x_5{r}_3+x_7{r}_5\\ x_5{r}_1+x_7{r}_3+x_9{r}_5\end{array}\right)+\left(\begin{array}{c}{x}_2{r}_2+x_4{r}_4\\ x_4{r}_2+x_6{r}_4\\ x_6{r}_2+x_8{r}_4\end{array}\right)=\left(\begin{array}{ccc}{x}_1& x_3& x_5\\ x_3& x_5& x_7\\ x_5& x_7& x_9\end{array}\right)\cdot \left(\begin{array}{c}{r}_1\\ r_3\\ r_5\end{array}\right)+\left(\begin{array}{cc}{x}_2& x_4\\ x_4& x_6\\ x_6& x_8\end{array}\right)\cdot \left(\begin{array}{c}{r}_2\\ r_4\end{array}\right). \end{gathered}$$

The resulting calculations can be implemented by a combination of the WM $F\left(3,3\right)$ using pixel brightness values $x_1,x_3,x_5,x_7,x_9$ and filter coefficients $r_1,r_3,r_5$, and $F\left(3,2\right)$ using pixel brightness values $x_2,x_4,x_6,x_8$ and filter coefficients $r_2,r_4$. Thus, in this case, instead of the $F\left(6,5\right)$ method for wavelet filtering with decimation, you can use the $F\left(3,5,2\right)=F\left(3,3\right)+F\left(3,2\right)$ method, where the third number in $F\left(3,5,2\right)$ denotes the degree of sampling rate reduction. In the general case of filtering the WM $F\left(n,r,d\right)$ and reducing the signal sampling frequency $d$ times, calculations are organized using a combination of methods according to the formula:

$$F\left(n,r,d\right)=s_2\cdot F\left(n,s_1+1\right)+\left(d-s_2\right)\cdot F\left(n,s_1\right),$$

(2)

where $s_1$ and $s_2$ are the partial quotient and the remainder of dividing $r$ by $d$, respectively. The notation of the WM $F\left(n,r,d\right)$ contains the size $n$ of the processed image fragments, the order $r$ of the wavelet used, and the decimation stride $d$.

The processing of a fragment with 14 $\left(n+r-1=14\right)$ pixels with a six-tap order filter $\left(r=6\right)$ is expanded to obtain nine values $\left(n=9\right)$ of a fragment of the processed image $Z$ at the output. For $d=3$, the values of $z_1,z_4,z_7$ are calculated:

$$\begin{gathered} \left(\begin{array}{c}{z}_1\\ z_4\\ z_7\end{array}\right)=\left(\begin{array}{cccccc}{x}_1& x_2& x_3& x_4& x_5& x_6\\ x_4& x_5& x_6& x_7& x_8& x_9\\ x_7& x_8& x_9& x_{10}& x_{11}& x_{12}\end{array}\right)\cdot \left(\begin{array}{c}{r}_1\\ r_2\\ r_3\\ r_4\\ r_5\\ r_6\end{array}\right)= \\ \left(\begin{array}{c}{x}_1{r}_1+x_2{r}_2+x_3{r}_3+x_4{r}_4+x_5{r}_5+x_6{r}_6\\ x_4{r}_1+x_5{r}_2+x_6{r}_3+x_7{r}_4+x_8{r}_5+x_9{r}_6\\ x_7{r}_1+x_8{r}_2+x_9{r}_3+x_{10}{r}_4+x_{11}{r}_5+x_{12}{r}_6\end{array}\right)=\left(\begin{array}{c}{x}_1{r}_1+x_4{r}_4\\ x_4{r}_1+x_7{r}_4\\ x_7{r}_1+x_{10}{r}_4\end{array}\right)+ \\ \left(\begin{array}{c}{x}_2{r}_2+x_5{r}_5\\ x_5{r}_2+x_8{r}_5\\ x_8{r}_2+x_{11}{r}_5\end{array}\right)+\left(\begin{array}{c}{x}_3{r}_3+x_6{r}_6\\ x_6{r}_3+x_9{r}_6\\ x_9{r}_3+x_{12}{r}_6\end{array}\right)=\left(\begin{array}{cc}{x}_1& x_4\\ x_4& x_7\\ x_7& x_{10}\end{array}\right)\cdot \left(\begin{array}{c}{r}_1\\ r_4\end{array}\right)+\left(\begin{array}{cc}{x}_2& x_5\\ x_5& x_8\\ x_8& x_{11}\end{array}\right)\cdot \left(\begin{array}{c}{r}_2\\ r_5\end{array}\right)+ \\ \left(\begin{array}{cc}{x}_3& x_6\\ x_6& x_9\\ x_9& x_{12}\end{array}\right)\cdot \left(\begin{array}{c}{r}_3\\ r_6\end{array}\right) \end{gathered}$$

The resulting calculations can be implemented by the F(3,2) WM using pixel brightness values of $x_1,x_4,x_7,x_{10}$ and filter coefficients of $r_1,r_4$, for values $x_2,x_5,x_8,x_{11}$ and coefficients $r_2,r_5$, as well as values of $x_3,x_6,x_9,x_{12}$ and coefficients of $r_3,r_6$. Thus, in this case, instead of using the $F\left(9,6\right)$ WM when filtering with a stride of 3, you can use the $F\left(3,6,3\right)=3F\left(3,2\right)$ method. In the special case when $d$ divides $r$, Formula (2) will take the form:

$$F\left(n,r,d\right)=d\cdot F\left(n,s_1\right).$$

(3)

The original signal is divided into two groups of samples (even and odd) during wavelet processing of the WM $F\left(n,r,d\right)$ images. In this case, the calculations are divided into two computational channels corresponding to even and odd signal samples. Multiplications of matrices $G{R}_L$ and $G{R}_H$ by the WM are performed a priori, once for each filter used, and do not require additional computational costs. The $B^{T}N$ multiplication is calculated before the calculations are divided into two channels, so it is the same for both wavelet filters used. Thus, the number of operations when processing wavelet images can be reduced by one calculation of $B^{T}N$ with the subsequent division of calculations into two channels. The elements of the transformation matrices $A^T$ and $B^T$ are also known in advance and consist of zeros, powers of twos, and numbers represented in binary notation as a set of ones. Multiplications by these elements can be represented as scalings and additions. For example, multiplying a number by $3={11}_2$ can be executed by moving the point of this number one bit to the right and adding it to the original number. Thus, all calculations according to Formula (1) are implemented using additions, with the exception of element-wise multiplication $\odot$. Element-wise multiplication is performed once for two $n\times 1$ matrices. The scheme for processing wavelet images with decimation for the WM is presented in Figure 2, where $Z_L$ and $Z_H$ are fragments of the image processed by the WM using wavelet filters $R_L$ and $R_H$, respectively.

D. Computationally Efficient Data Representation

All data in digital devices are stored with limited accuracy. Thus, it is necessary to quantize the coefficients of wavelet filters. For experiments, we use Daubechies wavelets db2 and db3. During wavelet processing of 8-bit images, the coefficients of the filters used are represented with an accuracy selected in accordance with the formula [17]:

$$f=10+\lfloor \sqrt{\frac{r}{4}}\rfloor ,$$

(4)

where $f$ is the capacity of the quantized wavelet filter coefficients without taking into account the sign bit and $\lfloor ·\rfloor$ is the rounding down operator. The original filter coefficients $F$ are scaled by $f$ bits and rounded up:

$$F^{\ast }=\lceil F·2^f\rceil ,$$

(5)

where $F^{\ast }$ is a quantized filter and $\lceil ·\rceil$ is the rounding up operator.

For example, the initial coefficients of the high-frequency wavelet filter db2 are:

$$\mathrm{HD}=\left(\begin{array}{cccc}\frac{-1-\sqrt{3}}{4\sqrt{2}}& \frac{3+\sqrt{3}}{4\sqrt{2}}& \frac{-3+\sqrt{3}}{4\sqrt{2}}& \frac{1-\sqrt{3}}{4\sqrt{2}}\end{array}\right).$$

According to Formula (3), for the considered wavelet filter, $f=11$. The coefficients of the wavelet filter are quantized using Formula (4):

$$\mathrm{HD}=\left(\begin{array}{cccc}-699& 1212& -324& -187\end{array}\right).$$

The resulting values are scaled and rounded down after WM wavelet filtering to compensate for the rounding error. Thus, the limited accuracy of data representation in the device memory will not have a significant impact on the quality of wavelet image processing.

Below, we present the results of implementing both considered approaches to wavelet image processing on FPGAs and an analysis of the results obtained.

3. High-Speed Implementation of the Discrete Wavelet Transform Using the Direct Method and the Winograd Method

Hardware modeling of the discrete wavelet transform using the DM and the WM with decimation was carried out on an FPGA in the Xilinx Vivado 2018.2 environment in the Verilog on FPGA Family Virtex 7 on the board “xc7vx485tffg1157-1” with the synthesis parameters “Vivado Synthesis Defaults” and implementation parameters “Vivado Implementation Defaults”, without using DSP blocks. Wallace tree structures were used as tools for implementing addition and multiplication, with further addition performed using carry-saving adders [18] and Kogge–Stone adders [19]. These adders perform calculations in the least amount of time [20]. The 8-bit pixels of the original image were used as input. Daubechies wavelets of four-tap and six-tap order were selected, the filter coefficients of which are quantized by 11 bits. The output data are 8-bit pixels of the processed image, since the WM receives several values of processed pixels in one iteration. The methods were evaluated based on the average resource costs for processing each pixel for a correct comparison. The simulation results are presented in

Table 1. Results of modeling a wavelet image processing device using the direct method and the Winograd method with averaged values for each pixel.

Tap	Method	Processed Image Fragment Size	Area, LUTs	Delay, ns	Power, W	Area-Delay Product
4	Direct	1	617.0	14.815	46.26	9140.86
	$F\left(2,4,2\right)$	2	692.5	9.732	62.25	6739.07
	$F\left(3,4,2\right)$	3	869.0	7.263	86.61	6311.55
	$F\left(4,4,2\right)$	4	1053.3	6.908	148.74	7275.85
	$F\left(5,4,2\right)$	5	1205.2	5.415	205.62	6525.92
6	Direct	1	903.0	16.730	78.77	15,107.19
	$F\left(2,6,2\right)$	2	916.5	10.222	80.19	9368.01
	$F\left(3,6,2\right)$	3	1062.0	7.786	126.07	8268.73
	$F\left(4,6,2\right)$	4	1175.5	6.747	179.33	7931.10
	$F\left(5,6,2\right)$	5	1125.0	5.706	176.98	6419.03

. The graphs in Figure 3, Figure 4, Figure 5 and Figure 6 show the hardware, time, and power consumption costs separately, as well as the area-delay product (ADP).

4. Discussion

The following conclusions can be drawn according to the results presented in Table 1 and Figure 3, Figure 4, Figure 5 and Figure 6:

• The hardware costs for wavelet processing of WM images compared to the DM increase from 12% (|692.5 − 617|/617 × 100% ≈ 12%) to 95% (|1205.2 − 617|/617 × 100% ≈ 95%), where the operator |·| is present to calculate the absolute value, and from 1.5% (|916.5 − 903|/903 × 100% ≈ 1.5%) to 30% (|1175.5 − 903|/903 × 100% ≈ 30%) using four-tap and six-tap wavelets, respectively.
• The computational delay in the wavelet processing of WM images compared to the DM is reduced from 34% (|9.732 − 14.815|/14.815 × 100% ≈ 34%) to 63% (|5.415 − 14.815|/14.815 × 100% ≈ 63%) and from 39% (|10.222 − 16.730|/16.730 × 100% ≈ 39%) to 66% (|5.706 − 16.730|/16.730 × 100% ≈ 66%) using four-tap and six-tap wavelets, respectively.
• The power consumption during wavelet processing of WM images compared to the DM increases from 35% (|62.25 − 46.26|/46.24 × 100% ≈ 35%) to 344% (|205.62 − 46.26|/46.24 × 100% ≈ 344%) and from 2% (|80.19 − 78.77|/78.77 × 100% ≈ 2%) to 125% (|176.98 − 78.77|/78.77 × 100% ≈ 125%) using four-tap and six-tap wavelets, respectively.
• The best efficiency in terms of the ADP is observed when using the F(3,4,2) and F(5,6,2) WMs using four-tap and six-tap wavelets, respectively.
• In general, the WM significantly speeds up calculations due to a moderate increase in hardware costs and power consumption.
• The increase in the speed of wavelet image processing will be insignificant compared to the increases in hardware costs and power consumption and the further increase in the size of the processed image fragments using the WM. This will also significantly increase the calculation error and degrade the quality of image processing.

Table 2. Comparison of wavelet transform methods.

Method	Year	Key Idea
Daubechies [21]	1992	Convolution-based direct method
Sweldens [22]	1995	Lifting scheme
Arts et al. [23]	2022	Algorithm with scale-independent and scale-dependent operation separation
Proposed	2023	Matrix computations using the Winograd method

presents a comparison of wavelet transform methods. In [21], a standard wavelet transform method was presented, using which hardware simulations were carried out and compared with the proposed method. The proposed method showed the best results in terms of the delay and overall device performance. Paper [22] describes a wavelet transform method called the lifting scheme. The peculiarity of this method is the sequence of the action steps: split, predict, and update. The lifting scheme allows a reduction in the hardware costs of the device, but at the same time increases the computation time. The method is applicable only to a limited set of wavelets. In [23], a fast continuous wavelet transform was implemented, and it requires large time and hardware costs to implement. The method developed in this paper is free of these disadvantages and is a promising digital signal processing tool for practical use and can be used in fusion problems of noise removal, compression, and combination of signals.

5. Conclusions

In this paper, calculations were carried out and Formulas (2) and (3) were derived, which implement wavelet filtering using the WM with decimation. The scope of application of the WM has been expanded to the case of downsampling a signal with an arbitrary stride during processing. The main result from hardware simulations of the developed approach on an FPGA is a reduction in the computational delay by up to 66%, with increases in hardware costs and power consumption of 95% and 344%, respectively, compared to the DM depending on the selected method parameters. The best efficiency in terms of the ADP was found for the $F\left(3,4,2\right)$ and $F\left(5,6,2\right)$ WMs using four-tap and six-tap wavelets, respectively. The developed approach can be used in image processing systems to improve the performance of modern microelectronic devices for image denoising and compression, as well as pattern recognition. A promising direction for further research is the implementation of the developed approach on ASIC and the use of modular computing for more efficient parallelization of calculations and an even greater increase in the device speed.