Leveraging Deep Convolutional Neural Network for Point Symbol Recognition in Scanned Topographic Maps

Abstract

Point symbols on a scanned topographic map (STM) provide crucial geographic information. However, point symbol recognition entails high complexity and uncertainty owing to the stickiness of map elements and singularity of symbol structures. Therefore, extracting point symbols from STMs is challenging. Currently, point symbol recognition is performed primarily through pattern recognition methods that have low accuracy and efficiency. To address this problem, we investigated the potential of a deep learning-based method for point symbol recognition and proposed a deep convolutional neural network (DCNN)-based model for this task. We created point symbol datasets from different sources for training and prediction models. Within this framework, atrous spatial pyramid pooling (ASPP) was adopted to handle the recognition difficulty owing to the differences between point symbols and natural objects. To increase the positioning accuracy, the k-means++ clustering method was used to generate anchor boxes that were more suitable for our point symbol datasets. Additionally, to improve the generalization ability of the model, we designed two data augmentation methods to adapt to symbol recognition. Experiments demonstrated that the deep learning method considerably improved the recognition accuracy and efficiency compared with classical algorithms. The introduction of ASPP in the object detection algorithm resulted in higher mean average precision and intersection over union values, indicating a higher recognition accuracy. It is also demonstrated that data augmentation methods can alleviate the cross-domain problem and improve the rotation robustness. This study contributes to the development of algorithms and the evaluation of geographic elements extracted from STMs.

1. Introduction

Scanned topographic maps (STMs) store and express geographic information that comprises important representations of landforms and features [1,2]. They are widely applied in the domains of resource development, engineering construction, and military defense [3,4]. Therefore, the extraction of geographic elements from STMs has attracted considerable attention [5,6]. In particular, the recognition and positioning of point symbols on STMs provide geographic and positioning information of geographic elements [7,8]. Examples of point symbols on an STM are shown in Figure 1; the following characteristics of point symbols pose significant challenges in their recognition [9]:

(1)

Point symbols are small, have similar shapes, and are generally composed of simple geometric patterns.

(2)

They are difficult to separate from other geographic elements, such as contour lines, using image segmentation [10] or other existing methods.

(3)

Depending on the map quality, the symbols may be blurred, deformed, or discontinuous after scanning.

A review of the literature indicated that several methods have been proposed to overcome these limitations, including template matching, statistical decisions, syntactic structures, and neural networks [11,12,13]. The most primitive methods are based on template matching. Although this method is easy to use, it is susceptible to background noise, resulting in poor recognition performance.

Moreover, improved methods based on template matching approaches have been used to identify point symbols. A fuzzy classification and identification method was proposed in [14] to recognize oil well identifiers in scanned petroleum geological maps. On the basis of the Hough transform algorithm, a method called the shear line segment generalized Hough transform (SLS-GHT), which outperforms other methods in complex scenarios, was proposed in [15] for recognizing point symbols in STMs. However, building the R-table of SLS-GHT is a complicated and time-consuming process with poor generalization ability. Additionally, Reiher et al. [16] recognized symbols using the Hausdorff distance and neural networks. Leyk and Chiang [6] first introduced the concept of geographic context for automated recognition of scanned maps. Pezeshk [17,18,19] has made many contributions to symbol recognition on STMs, mainly using a set of morphological operations. Although these classical methods have yielded results, they still have the following limitations:

(1)

Some STMs cannot be released publicly. Thus, the inadequate sample of point symbols lacks the necessary data support for the study.

(2)

Because symbols frequently overlap with other geographic elements, they are difficult to recognize entirely; thus, it is challenging to achieve both efficient and accurate recognition.

(3)

One centimeter on the topographic map (TM) represents several kilometers of the field. However, the positioning of point symbols in the existing methods is slightly shifted, and the determination of symbol anchors must be improved.

Therefore, we found that advanced models and strategies that are highly efficient, accurate, and robust to noise and other factors are required for point symbol recognition. Recently, with the development of computer vision, various deep convolutional neural networks (DCNNs) have been proposed for solving different image processing tasks, such as object detection [20,21] and semantic segmentation [22,23]. Deep learning has been applied for remote sensing object detection [24,25,26] and identifying dynamic changes in ground objects [27] with high accuracy.

In recent years, convolutional neural networks (CNNs) [28] have been widely used in the field of computer vision, and researchers have applied them to symbol recognition tasks and made significant progress. In [29], a deep transfer learning architecture was proposed for learning a symbol classifier using AlexNet and recognizing multiple point symbols simultaneously to increase the overall efficiency. In [30], a DCNN and graph convolutional network (GCN) were combined to extract and recognize geological map symbols. In [31], deep learning technology was applied to piping and instrumentation diagrams (P&IDs) to recognize symbols and text at an industrially applicable level. In such optical character recognition methods, the foreground and background differ considerably and mostly have a regular shape. However, point symbols on STMs are more abstract and stick to the background, differing in the research questions.

Inspired by the aforementioned studies and considering the characteristics of point symbols, we investigated the potential of object detection methods in point symbol recognition and developed a DCNN model for detecting symbols on STMs. Object detectors can be classified as one-stage or two-stage. Although one-stage detectors sacrifice some accuracy to achieve a high speed, they can achieve similar performance to two-stage detectors or even better performance in recent studies [32]. Therefore, the proposed method is based on one-stage detectors. There are two types of one-stage object detectors: you only look once (YOLO) [33,34,35] and single-shot detector (SSD) [36]. However, STMs differ from other types of images in that the contain a more singular shape of features and require more accurate geographic element recognition, for which traditional object detectors are not well-adapted. Studies [21,37] have indicated that object detectors primarily focus on detecting apparent or large objects and do not perform well in detecting small objects in images. Therefore, dilated convolution, which enlarges the receptive field, has been introduced for detecting small objects in images. Chen et al. [38] inserted a dilated convolution into YOLOv3, and achieved a 5.69% improvement in mean average precision (mAP) compared to the original model for their dataset. Li et al. [39] employed astrous spatial pyramid pooling (ASPP) [40] to extract multiscale features. Wang et al. [41] proposed a simple hybrid dilation convolution framework to alleviate the gridding problem using different dilation rates.

Point symbols and natural objects differ in that the symbols have a single shape and texture feature and stick to other background elements, making it difficult to distinguish between positive and negative samples. Therefore, we employed ASPP instead of spatial pyramid pooling (SPP) [42] in the YOLOv4 network to obtain multiscale information. The new network performed well in point symbol recognition and positioning tasks on STMs. Moreover, we constructed point symbol datasets for the supervised algorithm and proposed two data augmentations to improve the generalization ability of the model. We used the basemap in Cartographic symbols for national fundamental scale maps—Part 1:Specifications for cartographic symbols 1:500, 1:1000 and 1:2000 topographic maps for experiments and symbol recognition. The main contributions of this study can be summarized as follows:

(1)

STM point symbol datasets were constructed. Many scanned and vectorized map images were annotated, which comprised 1909 scanned map images and 2505 vectorized map images at different scales. We developed a DCNN model based on YOLOv4 for symbol recognition. The ASPP module, which enlarges the receptive field to obtain more information about the symbol object, was adopted to improve the accuracy and efficiency of symbol recognition.

(2)

To solve the cross-domain problem caused by datasets comprising maps of different styles and improve the model robustness, two data augmentation methods were designed: Gaussian blur combined with the color jitter method and small-angle rotation with an affine transformation. Additionally, to improve positioning accuracy, the k-means++ clustering method was adopted to generate anchor boxes that were more suitable for our point symbol datasets. A cut-stitch approach was designed for large-scale maps and its effectiveness was tested through several experiments on our dataset.

(3)

We achieved an mAP of 98.11% and a mean intersection over union (mIoU) of 0.876 for point symbol recognition on our test set. The proposed method is the first to employ DCNNs for recognizing point symbols, and it achieved a higher recognition and positioning accuracy than mainstream detectors and classical algorithms.

The remainder of this paper is organized as follows. Section 2 provides a detailed description of the constructed datasets. Section 3 describes the novel variant of YOLOv4, the k-means++ clustering algorithm adopted for point symbol recognition, and the two data augmentation methods used. Section 4 presents the experiments designed to comprehensively compare the proposed model with mainstream detection models and classical algorithms. Finally, Section 5 summarizes the study, highlights its limitations, and provides possible directions for future work.

2. Point Symbol Datasets

A dataset is vital for supervised learning algorithms and provides data support for research on point symbol recognition. Thus, we manually edited the scanned and vectorized map image datasets for the point symbol recognition task. We studied ten types of point symbols as an example, which are presented in

Table 1. The symbols and their identifiers.

The Symbols
The identifiers	I	II	III	IV	V	VI	VII	VIII	IX	X
The meanings	sanjiaodian	dulitianwendian	kuangjing	dianshifasheta	fadianchang	biandiansuo	shiyoujing	kexueguancezhan	jinianbei	shuichang

.

2.1. Scanned Map Dataset for Training and Testing

To train and test our model, we first constructed the scanned map dataset, which comprised 1909 images from a scale of 1:50,000 scanned maps and 3717 single symbols. The dataset was divided into three parts: training, validation, and test sets at a ratio of approximately 8:1:1. Figure 2 shows the statistical results for the training set and total set. There are differences in the numbers of various point symbols, leading to a slight class imbalance but no long-tail distribution. The weighted training method was used to solve this problem such that the model was more focused on a smaller number of symbols. Additionally, this dataset recorded the image categories and coordinates of the point symbols on the STMs. Specific examples are shown in Figure 3.

2.2. Vectorized Map Dataset for Training

The other point symbol sub-dataset comprised 2505 images from vectorized maps of different scales. It contained 7471 single symbols, and the numbers of various symbols are shown in Figure 4. The vectorized maps had different textures and clearer symbol images than the scanned maps, as shown in Figure 5. Similar to the scanned map dataset, this vectorized map dataset recorded the image coordinates and symbol categories. To evaluate the generalization ability of the deep learning method for different data sources, this sub-dataset was used as another training set but with varying map styles in the same symbol system. It complemented the symbol dataset of STMs and allowed map diversity.

3. Methods

Both a high detection accuracy and a low computational cost are crucial for practical applications of symbol recognition methods. The YOLOv4 model has been extensively applied in the field of object detection; therefore, we leveraged it to recognize point symbols on STMs. The detection targets in natural images are large and the texture varies widely between different categories, allowing the foreground and background to be easily distinguished, as shown in Figure 6. In contrast, the background of STMs is complex; the point symbols overlap with other geographic elements, and the symbols have similar color and texture information, making it difficult to distinguish between positive and negative samples. Moreover, on STMs, symbols are presented as small densely distributed targets with relatively low-resolution maps, which necessitates targeted improvements to the existing detection framework.

We aimed to comprehensively use dilated convolution in SPP, which uses multiple parallel convolution layers with different dilation rates to extract multiscale features to reduce the loss of feature map details, and proposed ASPP-YOLOv4 to address the problem of small object detection and the insufficient positioning accuracy, as illustrated in Figure 7.

We resized the image to a fixed size to use it as the input for the ASPP-YOLOv4 model. During forward propagation, feature maps of different scales were obtained from the CSPDarknet53 backbone network. Because the point symbols on STMs are small, we adopted the ASPP module and the path aggregation network (PANet) to fuse multiscale feature information to improve the model performance for symbol recognition and positioning tasks. The outputs were three YOLO heads, which were responsible for predicting objects on small, medium, and large scales. The prediction module comprised three parts: object bounding boxes, object categories, and confidence. We used the complete intersection over union (CIoU) loss to avoid the degradation of generalized IoU (GIoU) loss to IoU loss in certain situations. In addition, DIoU_non-maximum suppression (NMS) is employed in our network, which considers the center of the prediction box, resulting in a more accurate recognition of point symbols.

3.1. Dilated Convolution

The space between the standard convolution kernels is injected with voids, which is called a dilated convolution. Dilated convolution uses sparse sampling instead of downsampling, thus enlarging the receptive field without increasing the network depth, number of pooling layers, or number of parameters. The receptive field ($R{F}_n$) is calculated using the following equation:

$$R{F}_n=\left\{\begin{array}{cc}{k}_n,\hfill & n=1\hfill \\ R{F}_{n-1}+(k_n-1)\times S_{n-1},\hfill & n\ge 2\hfill \end{array}\right.$$

(1)

where $k_n$ denotes the size of the convolution kernel and $S_{n-1}$ denotes the stride of the previous layer of the network. Evidently, a larger stride corresponds to a larger receptive field. However, a larger stride size is not always preferable. Dilated convolution is a sparse sampling method, and when multiple dilated convolutions are superimposed, some pixels of the STM image are not used, which results in losses of continuity and relevance of information. Therefore, dilated and standard convolutions are used in parallel in the ASPP-YOLOv4 model.

3.2. Atrous Spatial Pyramid Pooling (ASPP)

During a single dilated convolution, the gridding issue causes local information loss, and long-distance information lacks relevance. Inspired by SPP (Figure 8), the ASPP module uses multiscale receptive fields to achieve a balance between global and local features during image processing. The ASPP module uses dilated convolution to replace the maximum pooling in the SPP. As shown in Figure 9, the first branch comprises $1\times 1$ standard convolution, which aims to maintain the original perceptual field; the second to fourth branches comprise depth-separable convolutions with different dilation rates, which aim to perform feature extraction for obtaining different perceptual fields; and the fifth branch comprises the global average pooling of the input to obtain global features. Finally, the feature maps of the five branches are stacked in the channel dimension to fuse information from multiple scales and upsample the desired spatial dimension bilinearly. The ASPP is more accurate and effective for small-object detection tasks involving the extraction of multiscale features.

3.3. Data Augmentation for Generalization Capability

During the training of the ASPP-YOLOv4 model, we applied data augmentation methods to extend the original dataset and improve the generalization and reliability of the model. For training on the scanned and vectorized map datasets, common data augmentation methods were used, such as random cropping, random horizontal and vertical flipping, and random conversion of the original image into a gray image. Furthermore, to enhance the robustness of the proposed symbol recognition model and simulate the distortions that occur during map scanning, such as small angular tilts, an affine transformation data augmentation method [43] was used for the scanned map dataset. Specific examples are presented in Figure 10. The data augmentation method that was only applied to the vectorized map dataset training, the Gaussian blur combined with the color jitter method, as shown in Figure 11. This is because scanned and vectorized maps have different styles, and the model recognition can be improved by transforming the style via image processing.

3.4. Anchor Boxes Based on K-Means++ Clustering

The label bounding boxes of the point symbols in the dataset varied in size. Anchor boxes closer to the symbol label bounding box in size correspond to more accurate detection. However, the anchor frame used by YOLOv4 is within the anchor box range calculated from the natural object dataset, which differs from our dataset. Therefore, we adopted the k-means++ clustering algorithm to obtain the best priors for the anchor boxes in the dataset, which uses the IoU score instead of the Euclidean distance function as an evaluation criterion, as indicated by the following equation:

$$d(box,centroid)=1-IoU(box,centroid)$$

(2)

where $box$ denotes the sample, $centroid$ denotes the center point of the cluster, and $IoU(box,centroid)$ denotes the overlap ratio of the cluster and center boxes.

Using the k-means++ clustering strategy, we experimentally obtained the changes in the average IoU score (avgIoU) corresponding to different numbers of anchors (k-values), as presented in

Table 2. The relationship between the number of anchors and avgIoU is obtained based on the k-means++ clustering algorithm.

k-Values	1	2	3	4	5	6	7	8	9
	62.53	33.43	27.40	25.38	21.43	21.44	19.40	19.41	19.40
		58.64	47.51	33.62	40.40	37.37	37.37	34.34	33.34
			67.72	48.48	33.62	34.63	27.54	27.54	26.54
Anchors				66.71	54.54	48.48	48.48	43.42	43.42
					71.77	59.59	40.72	52.52	37.67
						73.80	59.59	40.72	51.52
							74.79	62.62	62.61
								76.81	53.92
									79.78
avgIoU	0.598	0.698	0.745	0.766	0.799	0.818	0.831	0.843	0.852

. As the number of anchors increased, the avgIoU value continued to increased, but the growth rate decreased in later stages. Considering the complexity of the proposed network, we selected a k value of 9, and obtained the following anchor box values: $\left[\right(19\times 40),(33\times 34),(26\times 54),(43\times 42),(37\times 67),(51\times 52),(62\times 61),(53\times 92),(79\times 78\left)\right]$.

4. Experimental Results and Analysis

In this section, we describe the experiments used to compare the proposed ASPP-YOLOv4 model with mainstream object detectors and state-of-the-art methods with regard to the recognition accuracy, feature visualization, runtime, and generalization capability. We initialized CSPDarknet53 with the pre-trained model parameters from the PASCAL VOC dataset for training ASPP-YOLOv4. All models were implemented using the PyTorch framework on an NVIDIA Quadro P6000 GPU with 24 GB memory. We used the stochastic gradient descent (SGD) optimizer with a momentum of 0.937 to train all the networks. Additionally, the initial learning rate was set to 0.0001 and the step learning strategy was used. In the experiments, we used a frozen training strategy with the same dominance parameters to train over 1000 epochs. The models were trained on the dataset described in Section 2 and tested on the scanned map dataset. Classical algorithms were implemented using Python.

In addition, positioning, classification, and speed evaluations were performed to comprehensively assess the symbol recognition results. The positioning evaluation indicator is the mIoU, which is the area ratio of the intersection of the predicted and true-positive boxes to the result of their union. The precision, recall, F1 score, and mAP were used as performance metrics to evaluate the classification performance of the models. The average precision (AP) for symbol recognition was computed using an IoU threshold of 0.5. The average inference time, which is calculated as the total time spent for the symbol recognition process divided by the number of images detected, was used as a speed evaluation indicator.

4.1. Comparison of Different Object Detectors Recognition Models

In this section, the performances of four mainstream object detectors, faster R-CNN [38], VGG-SSD [36], YOLOv3 [33], and YOLOv4 [34] are compared with that of the proposed model to validate the proposed method.

Table 3. Experimental results of different models for symbol recognition.

Model	Precision	Recall	F1 Score	mAP	mIoU
Faster R-CNN	97.21	96.44	96.82	97.81	0.874
VGG-SSD	96.68	96.59	96.63	97.58	0.865
YOLOv3	92.89	95.68	94.26	97.35	0.824
YOLOv4	96.38	96.40	96.40	97.86	0.851
ASPP-YOLOv4	97.64	96.48	97.06	98.11	0.876

presents the symbol recognition accuracies of the different models tested on the scanned map dataset, with an input image size of 416 × 416 pixels. As shown, all the models achieved similar high accuracies, regardless of whether they are two-stage or one-stage detectors. The best result among the two-stage detectors is obtained by faster R-CNN, whereas the overall best result is obtained by the proposed ASPP-YOLOv4 model. The classification and positioning accuracies of ASPP-YOLOv4 were slightly dominant, as indicated by the mAP and mIoU values, respectively. In particular, the mIoU increased from 0.851 to 0.876 compared with the initial YOLOv4 model, indicating the effectiveness of introducing the ASPP module.

Because of the class imbalance in the symbol training set and the different structures of point symbols, the performances of the models differed for each class symbol, as shown in Figure 12. Regarding the detection performance for different classes, the proposed ASPP-YOLOv4 obtained higher AP values than others for most symbols, particularly for point symbols V and VIII. Moreover, the training set contained distinct symbol characteristics and several point symbols, I, II, IV, and X, and their AP values reached 100%. In some special cases, there were no performance improvements compared with YOLOv4, owing to the distribution of symbols. The recognition performance of all five models for symbols III, V, and VI was sub-optimal. This is because the textures of these symbols were not obvious and were easily disturbed by other background elements.

Qualitative comparisons were made to complement the quantitative results. STMs A, B, and C were selected to perform point symbol recognition using these five models, as shown in Figure 13, Figure 14 and Figure 15. In this study, all point symbols were directly recognized in these maps. Many contours merge with symbols on STMs, which makes recognizing point symbols challenging. According to the data presented in Table 3, the object detectors accurately recognized most symbols after training. Symbol IX was incorrectly detected, as shown in Figure 15b, which is consistent with the highest recall value for the VGG-SSD presented in Table 3. ASPP-YOLOV4 obtained better performance for the position of most point symbols, such as symbol III shown in Figure 13 and Figure 15, and symbol II shown in Figure 15. The relatively high confidence of the identifier indicates that ASPP-YOLOV4 achieved more accurate recognition.

Additionally, these qualitative comparisons are consistent with those shown in Figure 12. VGG-SSD and YOLOV3 could not recognize symbols III, V, and VI well because these point symbols are similar to geographical elements in maps, such as residential land and contours, and exhibited relatively low confidence, as shown in Figure 13b,c and Figure 14b,c. Moreover, the confidence of the proposed ASPP-YOLOV4 for symbol recognition in maps A, B, and C was close to 1, and it achieved the most accurate positioning of predicted boxes. These experimental results indicate that the proposed model has a higher recognition accuracy and less positioning error on STMs than the other detectors.

4.2. Feature Analysis Based on Gradient-Weighted Class Activation Mapping (Grad-CAM)

Deep learning is often considered a “black box” that lacks interpretability. To obtain more insight into the characteristics of the different models, Grad-CAM [44] was used to visualize the image regions of high interest in the model that contributed the most to the recognition decision. Therefore, we selected point symbol III in Table 1 as the study subject for the experiment. Figure 16 presents a comparison of the Grad-CAM results of three different models using YOLO heads: YOLOv4 [34], ASPP-YOLOv4, and YOLOv4-tiny [35]. The experimental results and analysis are presented below.

Compared with the YOLOv4 model, the ASPP-YOLOv4 model had a more concentrated area of interest in the symbol characteristics, particularly at the 52 × 52 and 26 × 26 scales. YOLOv4 did not effectively capture regions of interest at these two scales. In the case of YOLOv4-tiny, although only two heads are used for prediction, it is possible to precisely focus on symbolic features for small-scale symbolic targets. However, it deviated when the mesoscale prediction was concentrated on another symbol of a similar shape; hence, it was not as good as ASPP-YOLOV4 for capturing the valid area. This indicates the advantage of the ASPP structure, wherein the model obtains contextual information near the symbol by enlarging the receptive field, allowing the effective capture of the target symbol. Among the three different feature scales, the large-scale learns a large range of features, which is more conducive to global feature extraction, while the small-scale focuses on the features of the task, which aids in learning local features. Through the fusion of global and local semantic information, richer spatial contextual information is obtained for symbol target recognition, effectively improving the symbol recognition accuracy.

4.3. Comparison of Different Input Image Scales

To further demonstrate the feasibility of the proposed model, experiments were performed with different input image sizes: 320 × 320, 512 × 512, and 640 × 640 pixels. As shown in

Table 4. Performance of ASPP-YOLOv4 model with different input scales.

Input Sizel	Precision	Recall	F1 Score	mAP	MACs(G)	Inference Time (ms)
$320\times 320$	97.53	89.26	93.29	97.04	37.92	47.66
$512\times 512$	98.05	92.89	95.32	97.86	97.06	62.88
$640\times 640$	98.67	94.01	96.46	98.24	151.66	73.18

, when the image scale increased, the recognition performance of ASPP-YOLOv4 improved to a certain extent. The images at these three scales were identified by the model with mAPs of 97.04%, 97.86%, and 98.24%. However, as the image scale increased, the multiply accumulate operations (MACs) of the model also increased. The MAC increased from 18.9 to 75.8 as both the image width and height were increased from 320 to 640 pixels, and ASPP-YOLOv4 required a longer inference time than it did for small-scale images.

Additionally, qualitative experiments were performed to complement the quantitative results, as shown in Figure 17 and Figure 18. More symbols are recognized completely and accurately in the image of 640 × 640 pixels than in those at smaller scales, and the confidence is also higher. As shown in Figure 17a, the proposed model incorrectly recognized four point symbols; a symbol is missing in Figure 18a. When the input image size was increased to 512 × 512 pixels, only one symbol was not recognized, as shown in Figure 17b. All symbols of size 640 × 640 were recognized correctly and completely, as shown in Figure 17c and Figure 18c. As the input size increased, the position of the prediction box moved closer to the smallest bounding rectangle of the symbol, indicating that the positioning accuracy increased. The worst positional results were obtained for the image of 320 × 320 pixels, whereas better results were obtained for images of 512 × 512 and 640 × 640 pixels, as shown in Figure 17 and Figure 18, respectively.

The experimental results indicated that a larger image scale corresponded to a higher point symbol recognition accuracy. In engineering applications, we must balance the recognition accuracy and the detection speed to select the appropriate input image size.

According to these results, we expect that more GPU memory will be consumed for recognizing point symbols in large-scale maps. Therefore, a new method was developed for recognizing point symbols in large-scale maps that achieves better results in less time. We designed the image processing operation of “first cut, then zoom, detect, and finally stitch”, as shown in Figure 19. The entire STM was cropped to a size of 208 × 208 pixels, and zeros were filled in the width and height directions for the undersized part. The resulting images were enlarged two times to 416 × 416 pixels for use as input images. The categories and coordinates of the detected symbols were individually recognized by ASPP-YOLOV4, and then recorded and stored. Finally, the images were stitched together and coordinate conversion was performed. The recognition results were plotted and output.

This method was applied for symbol recognition in large-scale maps, and the results are shown in Figure 20, with a scale of 1241 × 955 pixels. All target symbols in these large-scale maps were recognized completely and accurately. Moreover, all target symbols, except one in the image that was skewed during scanning, were also accurately recognized, as shown in Figure 20b. Clearly, the proposed method is superior to the method of compressing images to a single size for recognition, which requires more computation. Therefore, the proposed method can be used in practical applications.

4.4. Classical Algorithms versus Learning-Based Models for Point Symbol Recognition

As described in Section 3, the ASPP-YOLOv4 method autonomously and comprehensively learns image features and directly outputs the categories and coordinates of the target symbols. The traditional approach for point symbol recognition involves color segmentation of the map followed by manual extraction of the areas around the symbols. Subsequently, template matching [15] or generalized Hough transform (GHT) [12] based on manually defined rules is used to recognize point symbols on the STMs. The experimental results of these algorithms are shown in Figure 21. It was concluded that classical algorithms face the following problems:

•

Long runtime for point symbol recognition. The existing methods require denoising and preprocessing of the map before the recognition of point symbols; next, the image must be segmented to extract possible target areas, which is time-consuming. Moreover, the point symbols may be erreneously removed during preprocessing. However, the proposed deep learning method comprises an end-to-end process, wherein the input RGB images go through the model to directly obtain recognition results, and good recognition performance is achieved. Moreover, the proposed method recognizes multiple point symbols simultaneously, overcoming the limitation of existing methods that individually recognize symbols. The inference times for different methods are presented in

Table 5. The inference time of different methods.

The Maps	Size	Template Matching (s)	GHT (s)	Ours (s)
1	1110 × 738	0.965	4.431	0.085
2	504 × 484	0.453	1.730	0.057
3	389 × 385	0.418	1.670	0.049

.

•

Relatively poor generalization capability. The conventional method can detect point symbols of the same color, size, orientation, and shape. If the paper TM is tilted during scanning, the recognition performance may be poor. As shown in Figure 22, assuming that the size, orientation, shape, or background of the symbols on the map change slightly, the point symbols may not be accurately recognized. In contrast, as described in Section 4, Parts C and E, the proposed model can accurately recognize point symbols that are rotated, scaled, or derived from different basemaps. Therefore, the proposed method has a good generalization capability.

•

Complexity of manually defined rules. Point symbols are approximated as comprising line elements, based on the skeleton information of the symbols, and artificially defined rules are represented in the parameter space to recognize point symbols. When symbols consist of multiple graphical elements, it is difficult to represent them accurately. Thus, the classical algorithms are unstable and highly complex.

4.5. Ablation Experiments

4.5.1. Direct Transfer Learning from Vector-Map to Scanned Map via Gaussian Blur and Color Jitter Augmentation

To evaluate the generalization ability of the proposed method for different map styles, ablation experiments were performed on our scanned map test set using a model trained on the vectorized map dataset containing different map styles. We expected the differences between the training and test sets to make it difficult for the model to accurately capture the target symbol. As shown in

Table 6. Recognition results of a model trained on a vectorized map dataset for our scanned map test set and whether the Gaussian blur combined with color jitter augmentation was used during training.

Gaussian Blur Combined with Color Jitter Method	Precision	Recall	F1 Score	mAP
×	97.10	48.05	58.10	76.40
√	96.72	65.61	74.43	80.09

, the recall for recognition was only 48.05%; therefore, the Gaussian blur must be combined with the color jitter method to transform the vectorized map into a scanned map. The experimental results indicated a significant improvement in the recognition accuracy via the transfer learning strategy, with increases of 17.56%, 16.33%, and 3.69% in the recall, F1 score, and mAP, respectively. However, a significant increase in the recall indicates that the model can recognize more symbols, to the extent that the accuracy decreases slightly. Overall, this method allowed the model to better recognize point symbols, as shown in Figure 23, with fewer missed symbols and a higher confidence. However, the 65.61% recall and 80.09% mAP for the vectorized map dataset indicate that the generalization ability must be improved.

4.5.2. Improve Model Rotation Robustness via Small-Angle Rotation Data Augmentation

During TM scanning, the map may be unaligned (Figure 17a), which affects the symbol-recognition results. The images in the test set were rotated by 3° and 5° to simulate a small-angle tilt of the scanned map, and the experimental results are presented in Figure 24. As shown in the graph, the AP for the symbols in categories III, V, VI, and VII decreased significantly, indicating that these categories were less rotationally robust. The results shown in Figure 25 are consistent with the statistical results: symbols III, VI, and VII were not correctly identified in Map I, which was rotated by 2.5° and 5°, whereas the other symbols in the map were correctly recognized. This was due to the structural characteristics of the point symbols, which are more robust when they are composed of circles or triangles.

To improve the rotational robustness of the model, we applied a data-augmentation method of small-angle rotation with an affine transformation to the training set. A range of 10° clockwise and counterclockwise rotations was set for the training set, and the obtained model was used to recognize our test set rotated by 5°. The experimental results are presented in

Table 7. Comparison of the results based on whether the small-angle rotation data-augmentation method was used during training, which was tested on our test set rotated by 5°.

Small-Angle Rotation	AP										mAP
Data Augmentation Method	I	II	III	IV	V	VI	VII	VIII	IX	X
×	79.01	91.33	35.90	69.99	40.93	14.58	17.92	46.32	77.03	72.56	54.56
√	100.0	99.74	92.63	99.91	92.22	88.32	93.38	92.96	99.88	100.0	95.90

. As shown, the mAP was significantly increased by this data-augmentation method, and a good AP was obtained for all types of symbols. Thus, the poor rotation robustness was addressed, improving the generalization of the model and its applicability in engineering.

5. Conclusions

In this study, the ASPP-YOLOv4 model was designed with a modified YOLOv4 framework, which was used to predict the categories and locations of point symbols on STMs. Compared with four classical detectors (faster R-CNN, VGG-SSD, YOLOv3, and YOLOv4) and two state-of-the-art methods (template matching and GHT), the proposed method focuses more on the target area, as indicated by the Grad-CAM visualization. It achieved significantly higher recognition and positioning accuracies (mAP of 98.11% and mIoU of 0.876), ranking first among all the methods. Additionally, considering the limited GPU memory and runtime, a cut-stitch method was developed for recognizing large-scale maps. The feasibility of this approach was demonstrated by the fact that all the target symbols in two large-scale maps were promptly and accurately recognized. Furthermore, the model was shown to have strong generalization ability. It can recognize point symbols that are rotated, scaled, or derived from different basemaps, addressing the limitations of existing methods.

Although the proposed ASPP-YOLOv4 model achieved good performance in point symbol recognition, it has several limitations. As indicated by the arrows in Figure 20b, the same symbol was recognized twice, because the symbols are exactly tessellated when cropping a large image. Therefore, the cut-stitch approach must allow further screening of the recognition results. Second, if the paper TM is folded during scanning, some point symbols on the STM may be distorted. In this case, the positioning of all symbols is incorrect because of the map folding, which results in the offset of the geographic feature coordinates after map vectorization. If the paper TM is folded during scanning, it must be rescanned. Finally, we focused on STM images, and the objects and results of the processing were image data, whereas the result of point symbol recognition is obtained as vectorized data. Therefore, converting existing results into vectorized data will be the focus of future work.

Another problem was that in the dataset used in this study, the point symbols all pointed to the north and had a regular structure, but the TM also contained undirected symbols. Our model needs advanced algorithms, such as rotation invariance, to solve the problem of undirected symbol recognition. Moreover, only a few point symbols are distributed on the STM, and those in some maps cannot be publicly published. Therefore, only 10 common point symbols were included in our dataset, as shown in Table 1. However, TMs contain more types of symbols; therefore, we will collect more samples of point symbols to enrich the dataset in a future study. Thus, we will provide sufficient data to support the digital production of STMs.