EfficientUNet+: A Building Extraction Method for Emergency Shelters Based on Deep Learning

Abstract

Quickly and accurately extracting buildings from remote sensing images is essential for urban planning, change detection, and disaster management applications. In particular, extracting buildings that cannot be sheltered in emergency shelters can help establish and improve a city’s overall disaster prevention system. However, small building extraction often involves problems, such as integrity, missed and false detection, and blurred boundaries. In this study, EfficientUNet+, an improved building extraction method from remote sensing images based on the UNet model, is proposed. This method uses EfficientNet-b0 as the encoder and embeds the spatial and channel squeeze and excitation (scSE) in the decoder to realize forward correction of features and improve the accuracy and speed of model extraction. Next, for the problem of blurred boundaries, we propose a joint loss function of building boundary-weighted cross-entropy and Dice loss to enforce constraints on building boundaries. Finally, model pretraining is performed using the WHU aerial building dataset with a large amount of data. The transfer learning method is used to complete the high-precision extraction of buildings with few training samples in specific scenarios. We created a Google building image dataset of emergency shelters within the Fifth Ring Road of Beijing and conducted experiments to verify the effectiveness of the method in this study. The proposed method is compared with the state-of-the-art methods, namely, DeepLabv3+, PSPNet, ResUNet, and HRNet. The results show that the EfficientUNet+ method is superior in terms of Precision, Recall, F1-Score, and mean intersection over union (mIoU). The accuracy of the EfficientUNet+ method for each index is the highest, reaching 93.01%, 89.17%, 91.05%, and 90.97%, respectively. This indicates that the method proposed in this study can effectively extract buildings in emergency shelters and has an important reference value for guiding urban emergency evacuation.

1. Introduction

Extracting buildings is of great significance for applications such as urban planning, land use change, and environmental monitoring [1,2], particularly for buildings in emergency shelters. This process helps improve disaster prevention and mitigation and other management capabilities [3]. An emergency shelter is a safe place for emergency evacuation and temporary dwelling for residents in response to sudden disasters such as earthquakes [4]. These temporary facilities mainly include open spaces, such as parks, green spaces, stadiums, playgrounds, and squares [5]. When disasters occur, buildings are prone to collapse and can injure people [6]. Some areas cannot be used for evacuation. Therefore, extracting buildings from emergency shelters has important guiding relevance in evaluating the emergency evacuation capabilities of shelters.

In the early days, the building footprint in emergency shelters was mainly obtained by manual measurement. The spatial resolution of satellite remote sensing images has reached the submeter level with the development of Earth observation technology. High-resolution remote sensing images have the advantages of rich ground object information, multiple imaging spectral bands, and short revisit time [7,8,9]. Thus, these images can accurately show the details of urban areas, providing critical support for extracting buildings. Despite the detailed information that these images provide, spectral errors, such as “intra-class spectral heterogeneity” and “inter-class spectral homogeneity”, exist [10]. These errors increase the difficulty of building extraction. Moreover, buildings have various features, such as shapes, materials, and colors, complicating the quick and accurate extraction of buildings from high-resolution remote sensing images [11,12].

The traditional methods of extracting buildings based on remote sensing images mainly include image classification based on pixel features and object-oriented classification. The extraction methods based on pixel features mainly rely on the information of a single pixel for classification; these methods include support vector machine and morphological building index, which are relatively simple and efficient to use [13]. However, they ignore the relationship between adjacent pixels and lack the use of spatial information of ground objects. They are prone to “salt and pepper noise”, resulting in the blurred boundaries of the extracted buildings [14]. Based on object-oriented extraction methods, pixels are clustered according to relative homogeneity to form objects for classification, utilizing spatial relationships or context information to obtain good classification accuracy [15]. However, classification accuracy largely depends on image segmentation results, and the segmentation scale is difficult to determine; thus, problems such as oversegmentation or undersegmentation are prone to occur [16], resulting in complex object-oriented classification methods.

Deep learning has a strong generalization ability and efficient feature expression ability [17]. It bridges the semantic gap, integrates feature extraction and image classification, and avoids preprocessing, such as image segmentation, through the hierarchical end-to-end construction method. It can also automatically perform hierarchical feature extraction on massive raw data, reduce the definition of feature rules by humans, lessen labor costs, and solve problems such as the inaccurate representation of ground objects caused by artificially designed features [18,19]. With the rapid development of artificial intelligence technology in recent years, deep learning has played a prominent role in image processing, change detection, and information extraction. It has been widely used in building extraction, and the extraction method has been continuously improved.

Convolutional neural network (CNN) is the most widely used method for structural image classification and change detection [20]. CNN can solve the problems caused by inaccurate empirically designed features by eliminating the gap between different semantics; it can also learn feature representations from the data in the hierarchical structure itself [21], improving the accuracy of building extraction. Tang et al. [22] proposed to use the vector “capsule” to store building features. The encoder extracts the “capsule” from the remote sensing image, and the decoder calculates the target building, which not only realizes the effective extraction of buildings, but also has good generalization. Li et al. [23] used the improved faster regions with a convolutional neural network (R-CNN) detector; the spectral residual method is embedded into the deep learning network model to extract the rural built-up area. Chen et al. [24] used a multi-scale feature learning module in CNN to achieve better results in extracting buildings from remote sensing images. However, CNN requires ample storage space, and repeated calculations lead to low computational efficiency. Moreover, only some local features can be extracted, limiting the classification performance.

Fully convolutional neural network (FCN) is an improvement based on CNN. It uses a convolutional layer to replace the fully connected layer after CNN; it also realizes end-to-end semantic segmentation for the first time [25]. FCN fuses deep and shallow features of the same resolution to recover the spatial information lost during feature extraction [26]. It is widely used in image semantic segmentation. Bittner et al. [27] proposed an end-to-end FCN method based on the automatic extraction of relevant features and dense image classification. Their proposed method effectively combines spectral and height information from different data sources (high-resolution imagery and digital surface model, DSM). Moreover, the network increases additional connections, providing access to high-frequency information for the top-level classification layers and improving the spatial resolution of building outline outputs. Xu et al. [28] pointed out that the FCN model can detect different classes of objects on the ground, such as buildings, curves of roads, and trees, and predict their shapes. Wei et al. [29] introduced multiscale aggregation and two postprocessing strategies in FCN to achieve accurate binary segmentation. They also proposed a specific, robust, and effective polygon regularization algorithm to convert segmented building boundaries into structured footprints for high building extraction accuracy. Although FCN has achieved good results in building extraction, it does not consider the relationship between pixels. It also focuses mainly on global features and ignores local features, resulting in poor prediction results and a lack of edge information. However, FCN is symbolic in the field of image semantic segmentation, and most of the later deep learning network models are improved and innovated based on it.

The UNet network model belongs to one of the FCN variants. It adds skip connections between the encoding and decoding of FCN. The decoder can receive low-level features from the encoder, form outputs, retain boundary information, fuse high- and low-level semantic features of the network, and achieve good extraction results through skip connections [30]. In recent years, many image segmentation algorithms have used the UNet network as the original segmentation network model, and these algorithms have been fine-tuned and optimized on this basis. Ye et al. [31] proposed RFN-UNet, which considers the semantic gap between features at different stages. It also uses an attention mechanism to bridge the gap between feature fusions and achieves good building extraction results in public datasets. Qin et al. [32] proposed a network structure U²Net with a two-layer nested UNet. This model can capture a large amount of context information and has a remarkable effect on change detection. Peng et al. [33] used UNet++ as the backbone extraction network and proposed a differentially enhanced dense attention CNN for detecting changes in bitemporal optical remote sensing images. In order to improve the spatial information perception ability of the network, Wang et al. [34] proposed a building method, B-FGC-Net, with prominent features, global perception, and cross-level information fusion. Wang et al. [35] combined UNet, residual learning, atrous spatial pyramid pooling, and focal loss, and the ResUNet model was proposed to extract buildings. Based on refined attention pyramid networks (RAPNets), Tian et al. [36] embedded salient multi-scale features into a convolutional block attention module to improve the accuracy of building extraction.

Most of the above methods of extracting buildings are performed on standard public datasets or large-scale building scenarios. They rarely involve buildings in special scenarios, such as emergency shelters. The volume and footprint of buildings in emergency shelters are generally small. For such small buildings, UNet [30] structure can integrate high- and low-level features effectively and restore fine edges, thereby reducing the problems of missed and false detection and blurred edges during building extraction. We use UNet as the overall framework to design a fully convolutional neural network, namely, the EfficientUNet+ method. We verify this method by taking an emergency shelter within the Fifth Ring Road of Beijing as an example. The innovations of the EfficientUNet+ method are as follows:

(1)

We use EfficientNet-b0 as the encoder to trade off model accuracy and speed. The features extracted by the model are crucial to the segmentation results; we also embed the spatial and channel squeeze and excitation (scSE) in the decoder to achieve positive correction of features.

(2)

The accurate boundary segmentation of positive samples in the segmentation results has always been a challenge. We weight the building boundary area with the cross-entropy function and combine the Dice loss to alleviate this problem from the perspective of the loss function.

(3)

Producing a large number of samples for emergency shelters within the Fifth Ring Road of Beijing is time-consuming and labor-intensive. We use the existing public WHU aerial building dataset for transfer learning to achieve high extraction accuracy using a few samples. It can improve the computational efficiency and robustness of the model.

This paper is organized as follows: Section 2 “Methods” introduces the EfficientUNet+ model overview, which includes EfficientNet-b0, scSE module, loss function, and transfer learning; Section 3 “Experimental Results” presents the study area and data, experimental environment and parameter settings, and accuracy evaluation and experimental results of the EfficientUNet+ method; Section 4 “Discussion” validates the effectiveness of the proposed method through comparative experiments and ablation experiments; Section 5 “Conclusion” presents the main findings of this study.

2. Methods

In this study, the method of deep learning was used to extract buildings in a special scene, emergency shelters. Given that the buildings in the emergency shelters are generally small, the use of high-resolution remote sensing images to extract buildings is prone to the problems of missed mentions and false and blurred boundaries. Based on the UNet model, EfficientUNet+, an improved building extraction method from high-resolution remote sensing images, was proposed. Beijing’s Fifth Ring Road emergency shelters comprised the research area. Figure 1 shows the technical route of this study.

2.1. EfficientUNet+ Module Overview

The UNet model is an encoder–decoder architecture, which consists of a compressed path for capturing context and a symmetric expansion path for precise localization. It uses skip connections to fuse the high- and low-level semantic information of the network [37]. Good segmentation results can be obtained when the training set is small. However, the original UNet model uses VGG-16 as the encoder, which has many model parameters, and the feature learning ability is weak. This study follows the model framework of UNet, applies EfficientNet in the UNet encoder, and proposes a deep learning-based method for extracting buildings in emergency shelters, namely, EfficientUNet+. Figure 2 shows the EfficientUNet+ module structure. The emergency shelters within the Fifth Ring Road of Beijing were taken as the research area to verify the effectiveness of the method in this study. The method is improved as follows. (1) The deep learning model used by the encoder is EfficientNet-b0, which is a new model developed using composite coefficients to scale the three dimensions of width/depth/resolution and achieves satisfactory classification accuracy with few model parameters and fast inference [38,39]. (2) The scSE is embedded in the decoder. Embedding spatial squeeze and excitation (sSE) into low-level features can emphasize salient location information and suppress background information; combining channel squeeze and excitation (cSE) with high-level features extracts salient meaningful information [40], thereby reducing false lifts of buildings. (3) The cross-entropy function is used to weigh the boundary area, improving the accuracy of building boundary extraction. The Dice loss is combined to solve the problem of blurred boundary extraction. (4) Given the small number of samples in the study area, a transfer learning method is introduced to transfer the features of the existing WHU aerial building dataset to the current Beijing Fifth Ring Road emergency shelter building extraction task, thereby reducing the labor cost of acquiring new samples and further improving the accuracy of building extraction.

2.2. EfficientNet-b0

In 2019, the EfficientNet model proposed by Google made a major breakthrough in the field of image classification. The network model was applied to the ImageNet dataset and showed superior performance. The model uses compound coefficients to scale the three dimensions of network depth (depth), network width (width), and input image resolution (resolution) uniformly; thus, the optimal classification effect can be obtained by balancing each dimension [41]. Compared with traditional methods, this network model has a small number of parameters and can learn the deep semantic information of images, greatly improving the accuracy and efficiency of the model [37,39]. EfficientNet also has good transferability [42].

The EfficientNet network consists of a multiple-module mobile inversion bottleneck (MBConv) with a residual structure. Figure 3 shows the MBConv structure. The MBConv structure includes 1 × 1 convolution layer (including batch normalization (BN) and Swish), k × k DepthwiseConv convolution (including BN and Swish; the value of k is 3 or 5), squeeze and excitation (SE) module, common 1 × 1 convolutional layer (including BN), and dropout layer. This structure can consider the number of network parameters while enhancing the feature extraction capability.

EfficientNet-b0 is a baseline architecture for lightweight networks in image classification [42]. As shown in

Table 1. Network structure of EfficientNet-b0.

Stage	Operator	Resolution	Layers
1	Conv 3 × 3	512 × 512	1
2	MBConv1, k3 × 3	256 × 256	1
3	MBConv6, k3 × 3	256 × 256	2
4	MBConv6, k5 × 5	128 × 128	2
5	MBConv6, k3 × 3	64 × 64	3
6	MBConv6, k5 × 5	32 × 32	3
7	MBConv6, k5 × 5	32 × 32	4
8	MBConv6, k3 × 3	16 × 16	1
9	Conv1 × 1&Pooling&FC	8 × 8	1

, EfficientNet-b0 consists of nine stages. Stage 1 consists of 3 × 3 convolution kernels with a stride of 2. Stages 2 to 8 consist of repeated stacking of MBConv, and the column parameter layers represent the number of times the MBConv is repeated. Stage 9 consists of a 1 × 1 convolution kernel, average pooling, and a fully connected layer. Each MBConv in the table is followed by number 1 or number 6. These numbers are the magnification factors. In particular, the first convolutional layer in the MBConv expands the channels of the input feature map to n times the original. k3 × 3 or k5 × 5 represents the size of the convolution kernel in the DepthwiseConv convolutional layer in MBConv. Resolution represents the size of the feature map output by this stage.

The EfficientNetb1-b7 series of deep neural networks chooses the most suitable one in width (the number of channels of the feature map), depth (the number of convolutional layers), and resolution (the size of the feature map) according to the depth, width, and resolution of EfficientNet-b0. The basic principle is that increasing the depth of the network can obtain rich and complex features. This approach can be applied to other tasks. However, the gradient disappears, the training becomes difficult, and the time consumption increases if the network depth is too deep. Given that the sample data are relatively small, we used EfficientNet-b0 as the backbone of the segmentation model.

2.3. scSE Module

scSE is a mechanism that combines spatial squeeze and excitation (sSE) and channel squeeze and excitation (cSE) [43]. It comprises two parallel modules, namely, the sSE and the cSE. Figure 4 shows the operation flow of the scSE module. This mechanism compresses features and generates weights on channels and spaces, respectively, and then reassigns different weights to increase the attention to the content of interest and ignore unnecessary features [44].

sSE is a spatial squeeze and excitation that improves the effectiveness of important features by assigning different weights to different spatial locations on the feature map. First, channel compression is performed on the feature map (C, H, W) using a 1 × 1 convolution block with channel C to transform this feature map (1, H, W). Then, the spatial location weights of the features on each channel are generated by normalization by the Sigmoid function. After the reconstruction of the spatial position relationship of the original feature map, a new feature map is finally generated. Equation (1) presents the calculation formulas.

$$U_{\mathrm{sSE}}=[\sigma (q_{1,1})u^{1,1},...,\sigma (q_{i,j})u^{i,j},...,\sigma (q_{H,W})u^{H,W}]$$

(1)

where $U_{\mathrm{sSE}}$ is the new feature map, $\sigma$ is the activation function, $q_{i,j}$ is the linear combination of spatial positions (i, j) under channel C, and $u^{i,j}$ is the spatial location of the feature.

cSE is a channel squeeze and excitation, which generates a channel-reweighted feature map by integrating the weight relationship between different channels. Thus, a channel-reweighted feature map is generated. First, the feature map (C, H, W) is generated by a global average pooling vector Z ∈ R^{C × 1 × 1}, where C, H, and W represent the channel number, height, and width of the feature map, respectively. The vector Z is operated by two fully connected layers to output a C × 1 × 1 vector. Then, a weight vector reflecting the importance of different channels is obtained through the Sigmoid function. Finally, the feature map is reweighted to generate a new feature map after feature filtering on the channel. Equations (2)–(5) present the calculation formulas [21].

$$u_c={\displaystyle \sum _{s=1}^{C’}{v}_c^s}\ast x^s$$

(2)

where $u_c$ is the output feature map; $C’$ and $C$ are the number of input and output channels, respectively; $v_c$ is the second two-dimensional spatial convolution kernel; ∗ means convolution operation; and $x^s$ is the sth input feature map.

$$z_c=\frac{1}{H\times W}{\displaystyle \sum _{i=1}^H{\displaystyle \sum _{j=1}^W{u}_c}}(i,j)$$

(3)

where $z_c$ is the generated vector through $u_c$ after global average pooling (squeeze operation), and H and W represent the height and width of the feature map, respectively.

$$s=\sigma (W_2\sigma (W_{1}z))$$

(4)

where $s$ is the vector output through $z$ after the excitation operation, $W_1\in R^{\frac{C}{r}\times C}$, $W_2\in R^{C\times \frac{C}{r}}$, and r is the scaling factor. Through the operation, $z$ converts to $\stackrel{\wedge }{z}$ and generates a new feature map as follows:

$$U_{\mathrm{cSE}}=[\sigma ({\stackrel{\wedge }{z}}_1)u_1,\sigma ({\stackrel{\wedge }{z}}_2)u_2,...,\sigma ({\stackrel{\wedge }{z}}_c)u_c]$$

(5)

2.4. Loss Function

The loss function is used to calculate the difference between the predicted value and the true value. The network model parameters are updated through the backpropagation of the error. The smaller the loss function value is, the better the model fitting effect is and the more accurate the prediction is [45]. The cross-entropy loss function is the most commonly used loss function in deep learning semantic segmentation. Equation (6) presents the formula of the two-category cross-entropy function.

$$Los{s}_{CE}=\frac{1}{N}{\displaystyle \sum _i-[y_i\cdot \log (p_i)+(1-y_i)\cdot \log (1-p_i)]}$$

(6)

where y is the prediction result and p is the ground truth. The weight of each pixel is equal by considering the cross-entropy function. The boundary area of the building is difficult to segment. We weigh the area’s cross-entropy loss from the perspective of the loss function. In backpropagation, the network is enhanced to learn the boundary regions. Equations (7) and (8) present the cross-entropy function formula for boundary weighting.

$$Los{s}_{CE\_BW}=Los{s}_{CE}\cdot Weight$$

(7)

$$Weight=\{\begin{array}{c}1,not boundary\hfill \\ w, boundary\hfill \end{array}$$

(8)

In this study, the value of w is 4. We introduce Dice loss to alleviate the imbalance in the number of positive and negative samples. Equations (9) and (10) present the final model loss function.

$$Loss=Los{s}_{CE\_BW}+Los{s}_{Dice}$$

(9)

$$Los{s}_{Dice}=1-\frac{{\displaystyle \sum _i}\mid p_i\cap y_i\mid }{{\displaystyle \sum _i}(\mid p_i\mid +\mid y_i\mid )}$$

(10)

2.5. Transfer Learning

Training often relies on a large amount of sample data to prevent overfitting in the process of training deep learning models. However, collecting sample data by visual interpretation requires a certain amount of experience and knowledge. It is also time-consuming and labor-intensive. In the case of a small number of samples, the existing data can be fully utilized through the transfer learning method. Transfer learning is further tuned by building a pretrained model on the source domain for feature extraction or parameter initialization and applying it to a related but different target domain [46,47]. Compared with training from scratch on a dataset with small sample size, transfer learning can improve computational efficiency and generalization of the model.

Given the complex, diverse, and changeable shapes and colors of target buildings, obtaining a large number of fine samples in the process of extracting buildings from emergency shelters within the Fifth Ring Road of Beijing is difficult even with manual visual interpretation, resulting in a small amount of sample data. Supporting the learning needs of a large number of network parameters is challenging. At present, most of the transfer learning research in the field of remote sensing uses ImageNet dataset for pretraining. However, ImageNet belongs to the field of natural images, and features such as resolution and depth of field are quite different from remote sensing data.

The WHU aerial building dataset is an open large-scale database often used for building extraction. The WHU aerial building dataset is very similar to the requirements of our task. Although the characteristics of the two building datasets are different, 8188 image data with a size of 512 × 512 pixels were obtained through WHU because of the relatively large amount of data in the WHU dataset. The characteristics of the buildings still have great versatility. Therefore, this study used the transfer learning method to pretrain the model based on the WHU aerial building dataset. The pretrained model parameters were used as the initial values of the Beijing building extraction model, effectively increasing the generalization ability of the model on the building dataset of the emergency shelters within the Fifth Ring Road of Beijing.

3. Experimental Results

3.1. Study Area and Data

3.1.1. Study Area

Beijing is the capital of China, covering an area of 16.4 km², with a resident population of 21.893 million [48]. It has become a distribution center of population, economy, and resources in the country. It also has an important geographical location in the country and even the world. Beijing is located at 39°26′N–41°03′N, 115°25′E–117°30′E, in the Yinshan–Yanshan seismic zone. It is one of the only three capitals in the world located in an area with a high earthquake intensity of magnitude 8. It is a key fortified city for disaster prevention and mitigation in the country. The central urban area of Beijing has dense buildings, a concentrated population, and the coexistence of old and new buildings. Once a disaster occurs, the damage caused by casualties and economic losses in this city is far greater than that in other areas. Therefore, the emergency shelters within the Fifth Ring Road of Beijing were selected as the research area, including parks, green spaces, squares, stadiums, playgrounds, and other outdoor open spaces. Among the emergency shelter types, the park exhibits large types and numbers of buildings. Thus, only the extraction of buildings in the park’s emergency shelters is considered in this study. According to the list of emergency shelters published by the Beijing Earthquake Administration and the list of registered parks published by the Beijing Municipal Affairs Resources Data Network, the Fifth Ring Road of Beijing has 118 parks that can be used as emergency shelters. Figure 5 shows the spatial distribution of park emergency shelter sites within the Fifth Ring Road of Beijing.

3.1.2. Dataset

The WHU aerial building dataset was used in this study to pretrain the model. Then, the created Google building dataset of emergency shelters within the Fifth Ring Road of Beijing was used to verify the effectiveness of the proposed method. Partial details of the two datasets are shown in Figure 6.

WHU aerial building dataset: The WHU dataset is divided into an aerial building dataset and a satellite building dataset. Given that the data used in this study are Google Images, the WHU aerial building dataset is similar to Google Image features. Thus, the standard open-source high-resolution WHU aerial dataset was used in this study as the training sample for transfer learning. The dataset was acquired in New Zealand, covering 220,000 buildings of different shapes, colors, and sizes, with an area of 450 km². The initial spatial resolution of the image is 0.075 m. Considering the memory and operating efficiency of the computer, Ji et al. [49] downsampled the spatial resolution of the image to 0.3 m and cropped the image in the area to a size of 512 × 512 pixels, forming an image dataset with 8188 images, including 4736 in the training set, 1036 in the validation set, and 2416 in the test set.

Google building dataset of emergency shelters within the Fifth Ring Road of Beijing: The dataset uses Google’s high-resolution remote sensing imagery with a spatial resolution of 0.23 m. We selected 21 typical parks with varying image sizes using expert visual interpretation to produce ground truth values for model training and evaluation. The 21 images and the corresponding ground truth values were cropped by the sliding window method to obtain 1110 image blocks with a size of 512 × 512 pixels. A total of 710 images were randomly selected as the training set for model parameter tuning, 178 images were used as the validation set for model parameter selection, and 222 images were used as the test set to evaluate the effect of the final model.

3.2. Experimental Environment and Parameter Settings

The experimental platform uses an Intel Core i7-8700@3.20 GHz 6-core processor, equipped with 32.0 G memory and an Nvidia GeForce RTX 3090. In terms of the software environment, we used the Windows 10 Professional Edition 64-bit operating system. The programming language is Python 3.7, the model building tool is PyTorch 1.7, and the graphics processing unit (GPU) computing platform is CUDA 11.0.

During model training, the batch size was set to 32, the initial learning rate was set to 0.001, the learning rate was adjusted by cosine annealing (the minimum learning rate is 0.001), the optimizer used Adam with weight decay (weight decay coefficient is 0.001), the number of iteration rounds was 120 epochs, and the model parameters corresponding to the rounds with the highest accuracy in the validation set were selected as the final model parameters. In addition, data augmentation operations of horizontal flip, vertical flip, diagonal flip, and 90-degree rotation were performed on the training data.

3.3. Accuracy Evaluation

This study used four indicators, Precision, Recall, F1-Score, and mean intersection over union (mIoU), to assess the building extraction accuracy and quantitatively evaluate the performance of the proposed method in extracting buildings [50,51]. Precision represents the proportion of the number of correctly predicted building pixels to the number of pixels whose prediction result is a building. Precision also focuses on evaluating whether the result is misjudged. Recall represents the proportion of the correctly predicted building pixels to the real building pixels. It focuses on evaluating whether the results have omissions. The F1-Score combines the results of Precision and Recall. It is the harmonic mean of Precision and Recall. The mIoU calculates the intersection ratio of each class and then accumulates the average. The mIoU also represents the ratio of the number of predicted building pixels to the intersection and union of the two sets of real buildings, that is, the overlap ratio of the predicted map and the label map. Equations (11)–(14) present the calculation formulas.

$$Precision=TP/(TP+FP)$$

(11)

$$Recall=TP/(TP+FN)$$

(12)

$$F1=2\times Precision\times Recall/(Precision+Recall)$$

(13)

$$mIoU=\frac{1}{k}{\displaystyle {\sum }_{i=0}^k[TP/(FN+FP+TP)]}$$

(14)

where TP means that the predicted building is correctly identified as a building; FP means that the predicted building is misidentified as a building; TN means that the predicted non-buildings are correctly identified as non-buildings; FN means real buildings are wrongly identified as non-buildings; and k is the number of categories.

3.4. Experimental Results

The EfficientUNet+ method proposed in this study was used to pretrain the model of the public dataset WHU aerial buildings. The experiments were conducted on the park emergency shelter buildings in the study area through the transfer learning method. The emergency shelter in Chaoyang Park has a large area and complex building types, shapes, and colors. Therefore, we took the emergency shelter in Chaoyang Park as an example. Figure 7 shows the results of the buildings extracted by the EfficientUNet+ method.

Five local areas of A, B, C, D, and E in the emergency shelter of Chaoyang Park were selected to see the details of the experimental results clearly. Figure 8 shows the original image, the corresponding ground truth, and extraction results.

Figure 8 shows that the outlines of the buildings in the emergency shelter are all extracted, the boundaries are complete and clearly visible, and only a few occluded buildings have broken boundaries. This observation shows that the EfficientUNet+ method proposed in this study can pay attention to the details in information while obtaining deep semantic information to achieve a complete building image, effectively extracting buildings in remote sensing images.

The four indicators, namely, Precision, Recall, F1-Score, and mIoU, were selected to evaluate the building extraction accuracy by the EfficientUNet+ method proposed in this study. The evaluation results are shown in

Table 2. Accuracy of EfficientUNet+ method for extracting buildings.

Precision	Recall	F1-Score	mIoU
93.01%	89.17%	91.05%	90.97%

.

Table 2 shows the quantitative results of using the EfficientUNet+ method to extract buildings from remote sensing images. The evaluation indicators reach approximately 90%; in particular, the Precision is 93.01%, the Recall is 89.17%, the F1-Score is 91.05%, and the mIoU is 90.97%. This finding indicates that the method can effectively extract buildings in high-resolution remote sensing images.

We further visualize the multi-scale architectural features extracted by the proposed model at different depths, as shown in Figure 9. From Figure 9b–f, we can see that the low-resolution architectural features are gradually refined as the feature resolution increases. The example in column (f) of Figure 9 illustrates that the semantic information of small-scale buildings cannot be captured by high-level features, because they occupy less than one pixel at low resolution.

4. Discussion

4.1. Comparison to State-of-the-Art Studies

To verify whether the proposed method performs better than other state-of-the-art methods, several deep learning methods commonly used in semantic segmentation and building extraction were selected as comparison methods, namely, DeepLabv3+, pyramid scene parsing network (PSPNet), deep residual UNet (ResUNet), and high-resolution Net (HRNet). Among these methods, the DeepLabv3+ method introduces a decoder, which can achieve accurate semantic segmentation and reduce the computational complexity [52]. The PSPNet method extends pixel-level features to global pyramid pooling to make predictions more reliable [53]. The ResUNet method is a variant of the UNet structure with state-of-the-art results in road image extraction [54]. The HRNet method maintains high-resolution representations through the whole process, and its effectiveness has been demonstrated in previous studies [55]. Some detailed images of emergency shelters were selected to compare the extracted accuracy and edge information clearly. Figure 10 shows the results of different methods.

Figure 10 shows that compared with other methods, the EfficientUNet+ method extracts almost all the buildings in the image and clearly shows the details, such as the edges and corners of the buildings, closely representing the real objects. The red box in Figure 10 shows that the above methods can extract the approximate location of the building. However, the EfficientUNet+ method can also extract the edge of the building, and its detail retention is higher than that of the other methods. The yellow box in Figure 10 shows that the results of DeepLabv3+, PSPNet, ResUNet, and HRNet methods have areas of misrepresentation and omission, whereas the EfficientUNet+ method can extract buildings more accurately than the other methods.

Four indicators were used to evaluate the extraction results of the EfficientUNet+, DeepLabv3+, PSPNet, ResUNet, and HRNet methods and to quantitatively analyze and evaluate the extraction accuracy. The results are shown in

Table 3. Accuracy comparison of the extraction results of different methods.

Methods	Precision	Recall	F1-Score	mIoU
DeepLabv3+ [52]	90.52%	87.15%	88.80%	88.92%
PSPNet [53]	76.40%	75.34%	75.87%	78.36%
ResUNet [54]	88.51%	80.72%	84.44%	85.16%
HRNet [55]	89.14%	83.43%	86.19%	86.63%
EfficientUNet+	93.01%	89.17%	91.05%	90.97%

. The accuracy comparison chart of the extraction results is shown in Figure 11 to intuitively compare the extraction accuracy of each method.

Table 3 and Figure 11 show that the accuracy of the EfficientUNet+ method for extracting buildings is 2.49%, 16.61%, 4.5%, and 3.87% higher than that of DeepLabv3+, PSPNet, ResUNet, and HRNet, respectively. The Recall of the EfficientUNet+ method is 2.02%, 13.83%, 8.45%, and 5.74% higher than that of DeepLabv3+, PSPNet, ResUNet, and HRNet, respectively. The F1-Score of the EfficientUNet+ method is 2.25%, 15.18%, 6.61%, and 4.86% higher than that of DeepLabv3+, PSPNet, ResUNet, and HRNet, respectively. The mIoU of the EfficientUNet+ method is 2.05%, 12.61%, 5.81%, and 4.34% higher than that of DeepLabv3+, PSPNet, ResUNet, and HRNet, respectively. In summary, the EfficientUNet+ method has the highest accuracy in each index, indicating that the EfficientUnet+ method proposed in this study can effectively extract the semantic information of buildings and improve the generalization ability of the model. The proposed method has certain advantages in extracting buildings from remote sensing images.

4.2. Ablation Experiment

4.2.1. scSE Module

The following ablation experiments were designed in this study to verify the effectiveness of adding the scSE module to the decoder trained by the model: (1) the network model with the scSE; (2) the network model without the scSE. Other experimental conditions are the same. The two methods were applied to the experiments on the building dataset of emergency shelters. The local details of the extraction results are shown in Figure 12. The accuracy comparison is shown in

Table 4. Accuracy comparison of extraction results of different decoders.

Method	Decoder	Precision	Recall	F1-Score	mIoU
EfficientUNet	Without scSE	90.81%	88.23%	89.50%	89.54%
EfficientUNet+	With scSE	93.01%	89.17%	91.05%	90.97%

.

Figure 12 shows that the EfficientUNet+ method with the scSE can basically extract all the buildings in the image, whereas the buildings extracted by the EfficientUNet method without the scSE have missed and false detection. Table 4 shows that adding the scSE to the decoder can improve the accuracy of model extraction of buildings. The extraction result analysis shows that the accuracy of each evaluation index after adding the scSE is improved. In particular, the Precision, Recall, F1-Score, and mIoU are increased by 2.2%, 0.94%, 1.55%, and 1.43%, respectively. The scSE added to the decoder enhances the feature learning of the building area, improves the attention of the features of interest, and suppresses the feature response of similar background areas, thereby reducing the false detection of buildings and improving the classification effect.

4.2.2. Loss Function

The following ablation experiments were designed in this study to verify the effectiveness of the boundary weighting in the loss function: (1) the cross-entropy function is weighted on the boundary area, and the Dice loss is combined; (2) the regular cross-entropy function and joint Dice loss are used. Other experimental conditions are the same. Figure 13 shows the local details of the extraction results.

Table 5. Comparison of the accuracy of prediction results of different loss functions.

Loss Function	Precision	Recall	F1-Score	mIoU
LossCE + LossDice	92.07	87.39	89.67	89.71
LossCE_BW + LossDice	93.01	89.17	91.05	90.97

shows the accuracy comparison.

Figure 13 shows the results extracted by the EfficientUNet+ method using boundary-weighted cross-entropy and Dice joint loss function. The boundary of the building is complete, and the edge is clearly visible. However, the buildings extracted by the EfficientUNet+ method without boundary weighting on the loss function have damaged and jagged boundaries. Table 5 shows that the area boundary weighting on the cross-entropy loss function improves the clarity, integrity, and accuracy of the edge details of the buildings in the result. The reason is that the boundary region has a substantial weight in backpropagation. The model also pays considerable attention, alleviating the boundary ambiguity problem of building extraction to a certain extent.

4.2.3. Transfer Learning

The following ablation experiments were designed in this study to verify the effectiveness of transfer learning: (1) the EfficientUNet+ method is first pretrained on the WHU aerial building dataset and then adopts transfer learning techniques; (2) the EfficientUNet+ method is directly applied to the Google emergency shelter building dataset. Other experimental conditions are the same. Figure 14 shows the local details of the extraction results.

Table 6. Accuracy comparison of prediction results with and without transfer learning.

Transfer Learning	Precision	Recall	F1-Score	mIoU
—	92.75%	88.92%	90.79%	90.73%
√	93.01%	89.17%	91.05%	90.97%

shows the accuracy comparison, where “√” indicates that the transfer learning technology is used for the experiment and “—” indicates that the transfer learning technology is not used for building extraction.

Figure 14 shows that the pretrained model EfficientUNet+ on the existing public WHU aerial building dataset is applied to the created Google building dataset using the transfer learning technology, thereby increasing the model’s ability to extract buildings and its generalization ability. Table 6 shows that the extraction accuracy of the transfer learning technology applied to the real object dataset is high, and the performance is stable. This finding shows that transfer learning can make full use of the existing data information, effectively solve the insufficient number of samples leading to model overfitting, and improve the generalization ability of the network. Thus, it can achieve satisfactory results in information extraction.

4.3. Efficiency Evaluation

We visualize the training loss versus epoch in Figure 15. It can be seen that the training loss of the proposed method decreases the fastest, far exceeding other comparison methods, which verifies its efficiency in the training phase. In addition, in order to verify the extraction efficiency of the proposed method, we count the operation time of the validation set, as shown in the table. It can be seen that the inference time and training time of the proposed method are 11.61 s and 279.05 min respectively, which are the shortest and the most efficient of all the compared methods.

Table 7. Operation time of buildings extracted by different methods.

Time	DeepLabv3+	PSPNet	ResUnet	HRNet	EfficientUNet+
Inference time	16.31 s	13.42 s	15.96 s	32.05 s	11.16 s
Train time	362.77 min	312.82 min	334.77 min	427.98 min	279.05 min

shows that the method proposed in this study can quickly extract the buildings in emergency shelters.

5. Conclusions

Buildings in special scenes, such as emergency shelters, are generally small. The extraction of such small buildings is prone to problems, such as integrity, misrepresentation and omission, and blurred boundaries. An improved deep learning method, EfficientUNet+, is proposed in this study, taking the emergency shelters within the Fifth Ring Road of Beijing as the research area. The effectiveness of the proposed method to extract buildings is verified. The following are the conclusions: (1) EfficientNet-b0 is used as the encoder, and the scSE is embedded in the decoder, which can accurately correct the feature map. Thus, the features extracted by the model are conducive to building extraction. (2) The joint loss function of building boundary-weighted cross-entropy and Dice loss can enforce constraints on building boundaries, making the building extraction results close to the ground truth. (3) Transfer learning technology can complete the high-precision extraction of buildings with few training samples in a specific scene background and improve the generalization ability of the model. The Precision, Recall, F1-Score, and mIoU of the EfficientUnet+ method are 93.01%, 89.17%, 91.05%, and 90.97%, respectively. Its accuracy is the highest among all evaluation indicators. This finding shows that the EfficientUnet+ method has suitable performance and advantages in extracting buildings in emergency shelters. The extraction results have guiding relevance in improving urban emergency evacuation capabilities and building livable cities.

However, the model sometimes misses extracting buildings that are obscured by trees. In the future, we will continue to optimize and improve the EfficientUNet+ method, try to extract buildings under different phenological conditions in summer and winter, and improve the accuracy and performance of remote sensing image building extraction. The method proposed in this study is suitable for optical remote sensing images. In the future, we will try to apply the proposed method to other datasets, such as side-scan sonar, to further verify the advantages of this method in small building extraction.