A Novel Image Recognition Method Based on DenseNet and DPRN

Abstract

Image recognition is one of the important branches of computer vision, which has important theoretical and practical significance. For the insufficient use of features, the single type of convolution kernel and the incomplete network optimization problems in densely connected networks (DenseNet), a novel image recognition method based on DenseNet and deep pyramidal residual networks (DPRN) is proposed in this paper. In the proposed method, a new residual unit based on DPRN is designed, and the idea of a pyramid residual unit is introduced, which makes the input greater than the output. Then, a module based on dilated convolution is designed for parallel feature extraction. Finally, the designed module is fused with DenseNet in order to construct the image recognition model. This model not only overcomes some of the existing problems in DenseNet, but also has the same general applicability as DensenNet. The CIFAR10 and CIFAR100 are selected to prove the effectiveness of the proposed method. The experiment results show that the proposed method can effectively reuse features and has obtained accuracy rates of 83.98 and 51.19%, respectively. It is an effective method for dealing with images in different fields.

1. Introduction

Deep learning [1] is a field of machine learning research. Its purpose is to train computers to complete human behaviors such as autonomous learning, judgment and decision making. It imitates the mechanisms of the human brain to explain data such as images, sounds and text [2]. With the development of the Internet and computer hardware, deep learning has an increasingly wide range of applications in image recognition [3]. In recent years, it has shown amazing capabilities.

Image recognition [4] is a technology that uses computers to analyze and process images to identify targets and objects in various patterns, based on the main features of the image. It is based on the main features of the image. There are many algorithms for image recognition, such as the improved differential evolution algorithm [5,6], enhanced MSI-QDE algorithm [7], neural network algorithm, and so on. Among them, the convolutional neural network [8] is one of the representative algorithms for image recognition using a neural network. The earliest emerging convolutional neural network algorithms are the Time Delay Neural Network [9] and LeNet-5 [10]. In 2012, AlexNet [11], which was designed by Hinton and his student Alex Krizhevsky, won the ImageNet competition and attracted widespread attention. In the ILSVRC competition in 2013, M.D. Zeiler and others improved AlexNet and innovatively added the convolution visualization function to form ZFNet [12], which won first place in the image classification task. At first, neural networks were mainly studied in terms of increasing depth, such as the top two GoogLeNet [13] and VGG [14] in the 2014 ILSVRC competition. VGG deepens the network depth to 19 layers and GoogLeNet to 22 layers. These have achieved good results. As the depth increases, the performance of the convolutional neural network structure becomes stronger and stronger. The problems of over-fitting, the disappearance of gradients, the huge amount of model parameters and difficulty in optimization are also becoming more and more prominent. Researchers have proposed different deep convolutional neural network model structures for different problems. ResNet [15] (Residual Net) was proposed by He. Its internal residual block uses skip connections to alleviate the gradient disappearance problem caused by increasing depth in deep neural networks, and it won the ILSVRC champion in image classification and target recognition in 2015. Since then, a variety of computer vision tasks have mainly used ResNet as the backbone network, while increasing the number of feature maps when increasing the dimension [16]. In 2016, the DPRN [17] (Deep Pyramidal Residual Networks) gradually increased the feature map dimension of all units and improved the residual module on the basis of ResNet to build a pyramid-shaped residual network. It has a good application in hyperspectral image classification [18]. The deep network architecture DenseNet [19] proposed by Huang in 2017 proposed a new approach for feature reuse. Each layer in the model gets additional input from all previous layers and passes its own feature map to all subsequent layers to make each layer densely connected to other layers, which not only slows down the phenomenon of vanishing gradients but also reduces the amounts of parameters. Hu proposed SENet [20] (Squeeze and Excitation Network). It is an architectural unit that aims to improve the representational ability of the network by dynamically recalibrating the channel features of the network. SENet performs a squeeze operation on the feature map obtained by convolution to obtain the global feature. Then, it performs an excitation operation on the global feature to learn the relationship between each channel and also obtains the weights of different channels. Finally, the result is multiplied by the original feature map to obtain the final feature. XIE [21] proposed the ResNeXt network, which replaced the three-layer convolution module of the residual network with a module of the same topology stacked in parallel to reduce the number of hyperparameters and improve computational efficiency. In 2019, Zhang [22] proposed a multi-feature weighted network MFR-DenseNet based on DenseNet for image classification. MFR-DenseNet improves the representational power of DenseNet by adaptively recalibrating channel feature responses and explicitly modeling the inter-dependencies between features of different convolutional layers. Sabour [23] proposed CapsNets (capsule network), the network is constructed by capsules, which is one of the new breakthroughs in deep learning methods. In 2020, Zhang proposed an improved ResNeSt [24] based on a residual network. The network designs a Split-Attention module that divides the feature map into several groups and finer-grained branches along the channel dimension. The feature representation of each group is represented by a weighted combination of its branches. By stacking Split-Attention, the ResNeSt network obtained by the Attention module is easier to transfer to other tasks than classification tasks [25,26,27,28,29,30,31]. In addition, some other methods or algorithms are also proposed, which can deal with images in recent years [31,32,33,34,35,36,37,38,39,40].

Most of these algorithms are implemented based on ResNet or DenseNet, and the image recognition methods formed by them all have their shortcomings. DPRN solves the shortcomings of ResNet to a certain extent, but the shortcomings of DenseNet are rarely mentioned. Therefore, aiming at the problems of insufficient feature utilization, single convolution kernel, and incomplete network optimization in DenseNet, a new convolutional neural network model based on DenseNet is designed by integrating DPRN and improving it. The experimental results show that the accuracy of the proposed model is better than the original model under the same conditions.

The main contributions of this paper are described as follows.

(1)

A novel image recognition method based on DenseNet and DPRN is proposed.

(2)

A new residual unit based on DPRN is designed, and the idea of a pyramid residual unit is introduced, which makes the input is greater than the output.

(3)

A module based on dilated convolution is designed. It increases the types of convolution kernels to enhance feature reuse and improves feature utilization efficiency.

2. Related Work

2.1. Residual Network

Under the premise that the neural network can converge, the traditional neural network will have problems with the increase in the network depth. That is, the performance of the network accuracy first gradually increases to a critical point. Then, it decreases rapidly, which is the problem of network degradation. In this regard, He from Microsoft Research proposed the Residual Network model, which provides two options for the features transmitted from the upper layer, residual mapping and identity mapping. The residual map is constructed based on a residual unit, and features are calculated through different convolution kernels and activation functions. The identity map directly adds the input of the unit to the output of the unit and activates it. It realizes the connection between the upper layer feature and the current layer feature in the form of a skip layer connection. After the network reaches the optimal level, if you continue to deepen the network, the residual map will be set to 0, leaving only the identity map to ensure that the network is always in the optimal state, and the performance of the network will not decrease as the depth increases. Therefore, the residual network alleviates the gradient vanishing problem of the deep network and can be easily implemented with the mainstream automatic differentiation deep learning framework. The Residual Network structure is shown in Figure 1. In Figure 1, X is the value output to the neuron from the previous layer.

2.2. Deep Pyramidal Residual Networks

Deep Pyramidal Residual Networks was proposed by Dongyoon Han in 2017. DPRN provides optimization of the residual unit model based on the residual network. It improves the module performance by removing the first linear rectification function (ReLU) layer and adding a batch normalization (BN) layer at the end. At the same time, DPRN proposes the idea of the bottleneck pyramid residual block. That is, the shape of the network architecture can be compared to the shape of the bottleneck pyramid, as shown in Figure 2 and Figure 3. That is to say, the number of feature map channels in a module decreases first, and then increases, so that the number of output features is greater than the number of input features.

The idea of the bottleneck pyramid-shaped module can be used by any network architecture to improve performance, and this design greatly improves the generalization ability of the model.

2.3. DenseNet

DenseNet is a deep network architecture proposed by Huang in 2017, which not only solves the problem of network degradation on the basis of the residual network but also increases the use of features of each layer. The DenseNet network is composed of multiple DenseBlock and Transition layers in series. Each DenseBlock is mainly composed of a 1 × 1 convolution kernel and a 3 × 3 convolution kernel, and the features of all the previous layers are densely connected with the latter layers to realize feature complexity, which greatly enhances the spread of features. The Transition layer is composed of a 1 × 1 convolution and pooling layer, which reduces the dimension of the input data and controls the feature shrinkage to prevent it from being too large. The transition layer structure is shown in Figure 4 and the DenseNet structure is shown in Figure 5:

Compared with other networks, DenseNet breaks away from the stereotyped thinking of deepening the number of network layers and widening the network structure to improve network performance. From the perspective of feature reuse, setting bypasses and dense connections at the expense of increasing memory not only greatly reduces the number of parameters of the network, but also alleviates the problem of gradient disappearance to a certain extent.

3. Image Recognition Method

3.1. Image Recognition

Image recognition is a technology based on the main features of an image and uses a computer to process and analyze the image to identify targets and objects in different patterns. It is a practical application of deep learning algorithms and an important part of the artificial intelligence field. It is of great significance to industrial production and people’s daily life. The convolutional neural network is a main feature extraction technology in the field of computer vision. Among them, image recognition using DenseNet has been a classic method in this field. However, this method also inevitably has some drawbacks. For the insufficient use of features, the single type of convolution kernel and the incomplete network optimization problems in densely connected networks (DenseNet), a novel image recognition method based on DenseNet and deep pyramidal residual networks (DPRN) is proposed in this paper. In the proposed method, a new residual unit based on DPRN is designed, and the idea of a pyramid residual unit is introduced, which makes the input greater than the output. Then, a module based on dilated convolution is designed for parallel feature extraction. Finally, the designed module is fused with DenseNet in order to construct the image recognition model. In addition, the entire DenseNet model is dominated by 3 × 3 convolution kernels and the types are relatively single. The overall structure still has room for optimization and the use of features can be further strengthened. Some modules in DenseNet are improved and new modules are added. The overall flow of the proposed method is shown in Figure 6.

Input the image to be recognized. A specific model through Dense Pyramidal Net is trained, and then the model is used for image recognition. In addition, the results can also be compared with the labels of the test images to obtain the accuracy of this image recognition. Next, the improvement points of the Dense Pyramidal Net will be gradually analyzed.

3.2. Module Improvements

The DenseBlock module is the main module in the DenseNet model, and its role is to extract features from images. However, there is a problem that the arrangement of various functional layers is not reasonable enough in this module, so there is still a lot of room for optimization. In this regard, this paper refers to DPRN and improves it by deleting some layers and adjusting the convolution kernel. The improved module is shown in Figure 7:

The literature [41] pointed out that the use of a large number of ReLU layers in each module may have some negative impacts on the overall performance of the module, and removing the first ReLU in each module can improve the performance to a certain extent. Therefore, the first ReLU layer of the original module is first deleted to appropriately enhance the performance. Secondly, according to the literature [13], it is shown that using a 1 × 1 convolution kernel to raise and lower the dimension and using a 3 × 3 convolution kernel to improve features requires fewer parameters than directly using a 3 × 3 convolution kernel for feature extraction while raising and lowering the dimension. More convolutions are superimposed on the receptive field of size, which can extract richer features and have higher computational efficiency. Therefore, this paper splits the 3 × 3 convolution block in the original Dense Block into a 3 × 3 convolution block with constant dimension and a 1 × 1 dimension reduction convolution block. The calculation formula of the convolution parameter quantity is described as follows.

$$p=K_1\times K_2\times \mathit{M}\times \mathit{N}$$

(1)

In the formula, p is the amount of convolution parameters. K₁, and K₂ are the length and width of the convolution kernel and they are both equal generally. M and N are the input and output feature map dimension vectors of the convolution layer, respectively. Formula (1) generally does not consider the bias parameters and the effect of the Batch Norm, thus simplifying the calculation. It can be seen from Formula (1) that in the case of using a pyramid structure, splitting the 3 × 3 convolution can reduce the amounts of parameters.

In addition, DenseNet mainly processes the input features through the Dense Block module, and there is only one convolution kernel for feature extraction, which is inevitably relatively single, and it is easy to lose the information in the picture. In order to make full use of the features without increasing too many parameters, a new dilated convolution block (DC Block) based on the dilated convolution method is designed. This problem is improved by adding this module in Figure 8.

In the DC Block, the input data is first normalized and activated by the BN layer and the ReLU layer, and then calculated by a 3 × 3 dilation rate [42] with a dilation rate of 2, and then normalized by a BN layer. Output to complete the calculation of the entire DC Block. The equivalent convolution kernel size calculation formula for dilated convolution is:

$$K=k+(k-1)\times (r-1)$$

(2)

where K indicates equivalent convolution kernel size, k indicates dilated convolution kernel size, and r is dilation rate. According to the calculation of Formula (2), the equivalent convolution kernel size of the 3 × 3 dilated convolution in the DC Block is 5, which reduces the parameters and can approximate the effect of the 5 × 5 convolution kernel, which not only changes the original DenseNet. It solves the problem of a single convolution kernel in the model and improves the richness of feature extraction in the model. It can read image features from different angles and make full use of different features for analysis.

3.3. Model Structure

Based on the improved DenseBlock module and DC Block module, the Dense Pyramidal Net network model structure is designed as shown in Figure 9.

Firstly, after the data is input, the 7 × 7 convolution layer is used to increase the dimension, and the size of the feature map is reduced at the same time and then the copied results are, respectively, entered into the improved DenseBlock and DC Block for parallel channel calculation. The DC Block can increase the receptive field, use a small number of parameters to achieve the effect of large kernel convolution, enrich the feature extraction results, and to a certain extent make up for the blank that the original model only has a single 3 × 3 convolution without other kernel convolutions. Image recognition. Finally, the results of the parallel computing are added, and the Cat splicing operation is performed with the features from the previous layers. The calculation formula of the output feature vector of the first layer is shown in Formula (3).

$${\mathit{x}}_{\mathit{i}}=\mathrm{Cat}({\mathit{x}}_1,{\mathit{x}}_2,\dots \dots ,{\mathit{x}}_{\mathit{i}-1},{\mathrm{H}}_{\mathit{i}}\left({\mathit{x}}_{\mathit{i}-1}\right)+{\mathrm{D}}_{\mathit{i}}\left({\mathit{x}}_{\mathit{i}-1}\right))$$

(3)

where, the Cat() function represents the concatenation operation; the H_i() function represents the nonlinear transformation of the i-th layer; and the D_i() function represents the dilated convolutional transformation of the i-th layer.

In order to further improve the compactness of the model, this model retains the Transition layer of the DenseNet, and adds the Transition layer after every certain number of DenseBlocks to reduce the number of feature map outputs. At the end of the model, a global average pooling layer is used to reduce dimensionality.

The global average pooling method is to average all the features of each feature map, and generate $k$ numbers of feature maps in the last convolutional layer of feature extraction; after the global average pooling layer, k numbers of 1 × 1 feature maps are obtained, Finally, these feature maps are input into the fully connected layer to obtain the confidence of $k$ numbers of categories, so as to obtain the classification result.

4. Experiment Results and Analysis

4.1. Experimental Setup

The experimental environment is described as follows. All network models in this experiment are based on the deep learning framework, and each model experiment is completed on a computer i7-11800H 8-core CPU, Windows10 operating system, 16 GB memory and NVIDIA GeForce RTX3060 Laptop GPU.

Parameter settings are given as follows. The networks in this experiment use the same parameter settings. On the data set, the learning rate size is set to 0.001, and the batch size is set to 100. The training epoch is set to 100, and the stochastic gradient descent optimizer with a momentum of 0.9 is used to optimize the network. All models use the ReLU activation function and use the cross-entropy loss function to calculate the loss.

The datasets are introduced as follows. The datasets used in this experiment are the public datasets of Cifar10 [43] and Cifar100 [44]. The Cifar10 dataset has a total of 60,000 RGB three-channel color images with a resolution of 32 × 32 pixels, which are divided into 50,000 training images and 10,000 test images, which can be divided into 10 categories; Cifar100 dataset and Cifar10 dataset are similar but the difference is that the Cifar100 dataset has 100 categories, each category contains 600 images, and each category has 500 training images and 100 testing images.

4.2. Ablation Experiment

In order to further explore the role of each improved part of the Dense Pyramidal Net model, an ablation experiment was designed. Each improvement is added sequentially from DenseNet to determine the impact of each part on the overall model. The results are shown in

Table 1. Comparison results of each part on Cifar10.

Part	Accuracy
DenseNet	77.10%
DenseNet with improved residual block	77.86%
Dense Pyramidal Net without DC block	79.78%
Dense Pyramidal Net	83.16%

.

It can be seen from Table 1 that the improved residual block has a partial impact on the original network, and the use of the pyramid structure increases the accuracy of model recognition. The parallel computing of the DC Block significantly enhances the model’s ability to recognize images.

4.3. Experimental Results and Analysis

In this paper, Densenet121 and Resnet101 are selected as comparison networks to verify Dense Pyramidal Net. The three networks are trained and tested on the cifar10 and cifar100 datasets, respectively. Accuracy and loss are mainly used as comparison metrics. In addition, the size of the model structure is also used as a basis for comparison. The test accuracies of the three networks on the cifar10 and cifar100 datasets are shown in

Table 2. Accuracy of the three algorithms on two datasets.

	Network	Dense Pyramidal Net	Densenet121	Resnet101
Dataset
Cifar10		83.98%	77.48%	73.74%
Cifar100		51.19%	46.51%	34.94%

. The parameters of the three models are shown in

Table 3. Three model parameters.

	Dense Pyramidal Net	Densenet121	Resnet101
Params(M)	18	8	45

.

It can be seen from Table 2 and Table 3 that the parameter amount of Dense Pyramidal Net is slightly larger than that of densenet121 and much smaller than that of resnet101 but the accuracy of Dense Pyramidal Net is ahead of them. The accuracy curves of each model on the cifar10 and cifar100 test datasets are shown in Figure 10 and Figure 11. As can be seen from the figure, the accuracy of the Dense Pyramidal Net network gradually slowed down around 20 rounds and finally reached a peak, and it was always in the lead in terms of accuracy, showing a stronger learning ability than other networks.

As can be seen from the Figure 10 and Figure 11, the accuracy of the Dense Pyramidal Net network gradually slowed down around 20 rounds and finally reached a peak, and it was always in the lead in terms of accuracy, showing a stronger learning ability than other networks.

The training losses of the three algorithms on the cifar10 and cifar100 datasets are shown in Figure 12 and Figure 13.

It can be seen from Figure 12 and Figure 13 that the loss curve of Dense Pyramid Net is between the two, gradually approaching flat, and finally reaching a value of 0. Combining the four pictures, Dense Pyramid Net not only has advantages over traditional algorithms in terms of accuracy but also the loss value curve is between the two algorithms. This shows that this model not only exhibits a high learning rate, that is, it can also quickly converge the loss value to get out of the underfitting state earlier and prevent overfitting. In this respect, the algorithm proposed in this paper is superior to the traditional algorithm and has strong robustness.

At the same time, the new algorithm ResNeSt50 is also compared, and the experimental results are shown in Figure 14. It can be seen that under the same conditions, the algorithm proposed in this paper is better than ResNeSt50 in 100 rounds of training.

The experimental results show that the optimized model structure, the use of parallel channels and convolution kernels of different sizes make the feature extraction more sufficient. The network training accuracy is also higher, and the anti-overfitting ability is stronger. The proposed Dense Pyramidal Net network has better performance when compared with DenseNet, the classic network rensnet101 and the new ResNeSt50.

5. Conclusions

In DenseNet, there are problems of insufficient feature utilization, a single type of convolution kernel, and incomplete network optimization. In this regard, this paper will improve DenseNet. First, by introducing the residual unit idea in DPRN, the ReLU layer and BN layer of the original module are appropriately increased or decreased to optimize the structure. Secondly, the module bottleneck pyramid structure is given, that is, the output is larger than the input bottleneck structure, which improves the training results. Then, a new module based on dilated convolution is designed to be computed in parallel with the original module, and the results are fused. Finally, experiments were performed on the CIFAR10 and CIFAR100 datasets, respectively, and obtained 83.98 and 51.19% accuracy. Experiments show that the proposed model can effectively reuse features and addresses some of the shortcomings in DenseNet. It has certain advantages in accuracy and efficiency by comparing with traditional networks.

Due to the grid effect of dilated convolution, that is, the convolution kernel is discontinuous, and there are some loopholes in identifying image features. This model still has room for improvement for pixel-by-pixel classification tasks. Hybrid Dilated Convolution (HDC) [45] is a new dilated convolution method that improves the shortcomings of dilated convolution. At the same time, this model does not discuss network security issues. In the future, it will be considered to combine the HDC with this model to reduce the loss of information and make the continuous use of feature maps more optimized. In addition, the model proposed in this article does not discuss security after being attacked by backdoors. At present, many scholars have also undertaken this work [46,47,48,49]. Related work in the field of security will also be carried out in the future.