AspectNet: Aspect-Aware Anchor-Free Detector for Autonomous Driving

Abstract

The anchor-free-based object detection method is a crucial part in an autonomous driving system because of its low computing cost. However, the under-fitting of positive samples and over-fitting of negative samples affect the detection performance. An aspect-aware anchor-free detector is proposed in this paper to address this problem. Specifically, it adds an aspect prediction head at the end of the detector, which can learn different distributions of aspect ratios between other objects. The sample definition method is improved to alleviate the problem of positive and negative sample imbalance. A loss function is designed to strengthen the learning weight of the center point of the network. The validation results show that the AP50 and AP75 of the proposed method are 97.3% and 93.4% on BCTSDB, and the average accuracies of the car, pedestrian, and cyclist are 92.7%, 77.4%, and 78.2% on KITTI, respectively. The comparison results demonstrate that the proposed algorithm is better than existing anchor-free methods.

1. Introduction

Vision-based object detection in traffic scenes plays a crucial role in an autonomous driving system. With the rapid development of deep learning in recent years, the performance of deep learning-based object detection has improved significantly. Two-dimensional object detection in an autonomous driving system can automatically recognize and locate the spatial position and category of the traffic object by learning the features of the traffic object and perceiving the traffic scenario to ensure the safety of the vehicle. It can be divided into anchor-based and anchor-free detection with the development of the convolutional neural network (CNN). In anchor-based detection, a set of preset size anchors are placed, and the positions and sizes of the anchors are regressed based on the ground truth. By contrast, anchor-free detectors directly detect objects without the anchors being set manually, which has a performance advantage over anchor-based detectors.

However, single-stage (dense) detectors that include anchor-based and anchor-free detectors still have many problems that affect performance, such as the definition of positive and negative samples, and the design of the loss function, which lead to sample imbalance problems. Additionally, the authors found that, in autonomous driving, the aspect ratios of different objects have different distributions, as shown in Figure 1. The inaccurate localization of objects can lead to the incorrect estimation of distance by the monocular ranging module, thus affecting the safe operation of autonomous driving. Anchor-based detectors (Faster R-CNN [1], and RetinaNet [2]) regress the object size using a large number of preset anchor boxes to make the network robust to aspect ratios. This mechanism has some drawbacks: (1) the anchor-based method performance is highly dependent on the selection of the number, size, and aspect ratio of the anchor. Anchor-based detectors need to fine-tune these hyperparameters. (2) In anchor-based detection, a set of anchor boxes needs to be densely placed. Most anchors are redundant and have little effect on model training. (3) Different detection tasks have different detection objects with large scale and shape variations, corresponding to the distribution of different anchor aspect ratios. It reduces the generalization ability of anchor-based detection method, for different anchor boxes need to be designed for new detection tasks. (4) In anchor-based detection, the computational cost increases due to the calculation of the intersection-over-union (IoU). Anchor-free detectors (CornerNet [3], CenterNet [4], and FCOS [5]) directly predict objects using key point or center point regression without preset anchor boxes. This mechanism can eliminate the hyperparameters related to the anchor, achieve similar performance to anchor-based detectors with less computational cost, and achieve better generalization ability. However, in regression using key points or the center point, prior knowledge of the aspect ratio is lacking, which increases the model training time required to fit the distribution.

Additionally, ATSS [6] indicated that the performance gap between anchor-based and anchor-free detectors is mainly affected by the definition of positive and negative samples. Inappropriate sample definition and sampling methods aggravate the imbalance of positive and negative samples. In Figure 2, the yellow box is the ground truth, the red box is the positive sample, and the remaining gray boxes are negative samples. Each point on the feature map generates multiple anchor boxes in anchor-based methods, such as YOLO v3 [7]. YOLO v3 only uses the anchor box with the maximum IoU value for the ground truth as the positive sample, and the remainder of the anchor boxes as negative samples. By contrast, anchor-free methods, such as CenterNet, only use the center point of the object as the positive sample and the remaining points as negative samples. The under-fitting of positive samples and over-fitting of negative samples make the model unable to thoroughly learn the features of objects, which affects the detection performance.

In this paper, an aspect-aware anchor-free detector called AspectNet is proposed to solve the above problems. It supervises the object width and height by adding an aspect prediction head at the end of the detector, which can learn different distributions of aspect ratios between different objects. Simultaneously, a new sample definition method and a loss function are proposed to alleviate the problem of sample imbalance.

The contributions of this paper are as follows:

(1)

A novel aspect-aware anchor-free detector is proposed to fit the distribution of the aspect ratios of different objects to alleviate the influence of variable scales on model robustness.

(2)

The sample definition method is improved to alleviate the problem of positive and negative sample imbalance.

(3)

A loss function is designed to strengthen the learning weight of the center point of the network, which solves the imbalance problem.

The remainder of this paper is structured as follows: In Section 2, recent advances in object detection methods are discussed. Section 3 contains information, including specific details and improvement methods. In Section 4, the implementation of the proposed method and its comparison with existing approaches are discussed. In Section 5, the proposed method is summarized, and the future research direction is presented.

2. Related Work

Traditional object detection uses HOG [8] or DPM [9] to extract the image features, then feeds the features into a classifier, such as SVM [10]. Because of its low performance, it was replaced by deep learning convolutional networks in the deep learning era. Generally, deep CNNs can be roughly categorized into two types: anchor-based approaches and anchor-free approaches.

2.1. Anchor-Based Approaches

Anchor-based approaches can be divided into two main types of pipelines: multi-stage detection and single-stage detection. Multi-stage detection methods filter regions of interest in the image and extract the foreground area from preset dense candidates, using region proposal algorithms [11,12] in the first stage. The bounding boxes of objects are constantly regressed in the subsequent stages, such as the R-CNN series [1,11,12,13], which is time consuming. Complex multi-stage models cannot meet the requirements of real-time detection. Different from multi-stage methods that use traditional sliding windows and proposals, single-stage detection methods inherit the ideas of anchor boxes but directly detect the object and regress the bounding boxes in a single-shot manner without any region proposals, which avoids the repeated calculation of the feature map and makes the anchor boxes directly on the feature map, which speeds up the detection process dramatically, for example, YOLOv3, RetinaNet, and SSD [14].

These methods need to preset a large number of anchors manually, which increases the calculation in the network, and the excessive number of samples aggravates the imbalance of positive and negative samples, as described above. Additionally, the misaligned problem between anchors and features affects the network’s performance. In previous studies, researchers often used IoU to determine the samples. For example, if the IoU between the anchor boxes and ground truth boxes of a sample is in [0.5, 1], then the sample is positive, whereas if the IoU is in [0, 0.5), then the sample is negative. The anchor with the maximum ground truth IoU is selected for matching. These hyperparameters need to be manually fine-tuned for different detection tasks, and detection tasks are susceptible to hyperparameter selection. The matching definition of positive and negative samples has a significant impact on the model’s performance.

2.2. Anchor-Free Approaches

Anchor-free approaches can directly detect objects without preset anchors and can be divided into two directions: key point-based methods and center-based methods. Most are single stage because both methods are dense detectors. Key point-based methods use multi-key point joint expression to predict objects. For example, ExtremeNet [15] uses the four extreme points and the center point to make predictions. RepPoints [16] adaptively learns a set of points to represent boxes. FoveaBox [17] uses the center point, and upper left and lower right corners. PLN uses the center point and four corner points. CornerNet uses a pair of corner points to regress the bounding anchor based on key point-based methods. However, it requires an extra distance metric to store the pair of corner points that belong to the same object, which requires complicated post-processing. Compared with key point-based methods, structure center-based methods are more concise and can achieve better performance [18]. Therefore, the proposed model AspectNet uses the center-based approach to detect objects. Center-based methods use the center point for fitting to predict objects. YOLOv1 [19] uses points near the object’s center to regress the bounding box instead of using anchor boxes. However, YOLOv1 has the characteristics of high precision and low recall because it only uses a small number of positive samples to learn features, and was quickly replaced by the anchor-based detector YOLOv2 [20]. CenterNet models the bounding box regression problem as the problem of learning the center point and distance between the center point and the corresponding bounding box, and directly uses the center point regression method to predict objects without NMS for post-processing. The structure is simple and effective. However, CenterNet’s positive samples are only defined as the center point of the ground truth, and the remaining points are negative samples. This simple approach to defining positive and negative samples causes CenterNet to have a severe imbalance of positive and negative samples, which also causes the overfitting of regression branches. FCOS is a pixel-wise object detection algorithm based on FCN [21], which alleviates the problem of positive and negative sample imbalance by adding a centeredness branch. It uses the feature pyramid network (FPN) for multi-scale prediction. FCOS defines the points in the bounding boxes of the object as positive samples, and the remaining points are negative samples. Compared with CenterNet, FCOS has a larger number of positive samples. However, FCOS does not define the ignored area of the sample, and the focal loss makes FCOS pay more attention to the noisy points around the bounding box. Additionally, the position of the bounding box edge is used as a positive sample for which it is difficult to obtain accurate results for prediction.

The excessive attention paid to difficult negative samples makes the loss unstable and difficult to decrease, hence not robust. The proposed method alleviates this problem by redefining sample selection and redesigning the loss function. Additionally, the aspect ratio branch of the proposed method can make up for the lack of prior knowledge of the aspect ratio in the anchor-free method.

3. Proposed Method

The proposed structure of AspectNet is shown in Figure 3. Different from other anchor-free models, an aspect-aware prediction head is designed and added to the end of the detector connecting the regression branch and classification branch to predict the aspect ratio of bounding boxes. The sample definition method and loss function of AspectNet are proposed to alleviate the sample imbalance problem.

3.1. Aspect Prediction for AspectNet

The unstable prediction of the aspect ratio of the bounding box can lead to wrong judgments by the autonomous driving decision-making module, as shown in Figure 4. Different objects have different aspect ratio distributions. For example, the expected value of the aspect ratio of the bounding box of the pedestrian is larger than that of the bounding box at the rear of the vehicle in front. The under-fitting of the model to the aspect ratio distribution of different objects makes model prediction unstable.

An aspect-aware head is added at the end of the detector to predict the aspect ratio of objects that concatenates the classification and regression branches. This can solve the robustness problem mentioned above, as shown in Figure 3. The aspect-aware head improves the localization ability of the network. By contrast, learning the distribution of different aspect ratios can improve the model’s classification ability.

Specifically, by labeling the aspect ratio of objects, the model can learn the aspect ratio distribution of each specific category. Pedestrians have different weights, and the mean distribution is higher. Cyclists have a large distribution variance due to their different riding postures. Although vehicles have different viewing angles (e.g., aspect ratios of frontal and side views of vehicles are different), the overall aspect ratio distribution is lower than pedestrians and cyclists. By distinguishing the aspect ratio distribution of different categories, the model can locate the position of objects and classify them more accurately. During training, the aspect prediction head is trained synchronously with the other two branches and indirectly improves model performance by backpropagating the weight updates to these two branches so that the model can extract the aspect ratio features of the objects. During inference, the confident detection score of the predicted objects adds to the disturbance to the aspect ratio based on the existing predicted classification probability. The detection confident $S_i$ score is defined as

$$S_i=p_i^{1+\left|As{p}_i-As{p}_i^{reg}\right|},$$

(1)

where $As{p}_i$ is the predicted aspect ratio of the bounding box $i$, $As{p}_i^{reg}$ is the aspect ratio calculated from predicted position information from the regression branch, $p_i$ is the predicted classification probability. The larger the aspect ratio difference between the aspect-aware head and the regression branch, the smaller the confidence of the predicted bounding box.

The aspect ratio $Asp$ is defined as

$$Asp=\frac{y_2-y_1}{x_2-x_1},$$

(2)

where $x_2,y_2$ are the coordinates of the lower right corner point of the bounding box, and $x_1,y_1$ are the coordinates of the upper left point. The feature maps are concatenated from the classification branch and regression branch. The entire aspect prediction head only consists of two convolution layers to keep the efficiency achieved by the RELU [22] activation layer.

3.2. Sample Definition Method

The single-stage detector has always been plagued by the imbalance of positive and negative samples. The reason that the accuracy of the two-stage detector is higher than that of the single-stage detector is because the two-stage detector uses RPN or other selection methods in the early stage to alleviate the imbalance of positive and negative samples. The performance gap between anchor-free and anchor-based detectors is caused by the difference in the sample definition method of positive and negative samples. Therefore, selecting an appropriate method for determining positive and negative samples is the key to designing object detectors. Most center-based anchor-free methods select one ground truth center point as a positive sample or select all points in the ground truth box as positive samples. The former causes serious imbalance problems, whereas the latter causes the model to focus too much on complex samples, which results in the loss being unstable and difficult to decrease. In this paper, a sample definition method is proposed based on a center-based anchor-free detector for object detection tasks in autonomous driving.

All ground truth center points $p=\left(\frac{x_1+x_2}{2},\frac{y_1+y_2}{2}\right), x_i\in ℝ, y_i\in ℝ$ are mapped to a heatmap $Y\in {\left[0,1\right]}^{\frac{W}{R}\times \frac{H}{R}\times C}$, where W and H are the width and height of the input image $I\in {ℝ}^{W\times H\times 3}$, respectively, R is a down-sampling factor, and C is the number of categories that use an aspect-aware Gaussian kernel $Y_{xyc}$ given as follows and demonstrated in Figure 5:

$$Y_{xyc}=\exp \left(-\frac{2h^2{\left(x-{\tilde{p}}_x\right)}^2+2w^2{\left(y-{\tilde{p}}_y\right)}^2}{{\left(wh{\sigma }_p\right)}^2}\right),$$

(3)

where ${\sigma }_p$ is an object size-adaptive standard deviation, w and h are the width and height of the object bounding box, respectively, and $\tilde{p}=\left|\frac{P}{R}\right|$ is a low-resolution equivalent of each ground truth center point $p=\left(\frac{x_1+x_2}{2},\frac{y_1+y_2}{2}\right)$. $Y_{xyc}$ represents the degree of influence of this point on the center point of the object. $Y_{xyc}=1$ corresponds to the center point of the object, whereas $Y_{xyc}=0$ is the background. Positive samples are defined as the points $Y_{xyc}\ge {\tau }_1$ and negative samples as the points $Y_{xyc}\le {\tau }_2$. Points ${\tau }_2<Y_{xyc}<{\tau }_1$ are ignored sample points, where ${\tau }_1$ and ${\tau }_2$ are hyperparameters that control the number of positive samples. In this paper, ${\tau }_1=0.5, \mathrm{and} {\tau }_2=0.1$ are chosen. By proposing the positive sample area, the ignored sample area, and the negative sample area, our model can effectively pay attention to the relationship between the features of the object itself and the background without paying too much attention to the edges of the object, which can also avoid the overfitting of object bounding box edges. The definition of positive samples is also related to the aspect ratio of objects, making the sample definition method more efficient and concentrated. Thus, the problem of the positive and negative sample imbalance can be alleviated.

3.3. Loss Function

The loss function of the proposed method can be divided into three parts that correspond to three heads: the classification part, regression part, and aspect-aware part. The classification loss function is aspect-aware logistic regression based on the focal loss. As shown in the proposed sample definition method, $Y_{xyc}$ is an aspect-aware Gaussian kernel, indicating the aspect ratio of the bounding box. The heatmap is a pixel-wise decrease from the center point of the bounding box to the surrounding region, and the decreasing speed is related to the aspect ratio.

The classification loss function is defined as

$${ℒ}_{cls}=\frac{-1}{N_{Pos+Neg}}{{\displaystyle \sum }}_{x,y,c}\{\begin{array}{c}{\left(Y_{xyc}-{\widehat{Y}}_{xyc}\right)}^{\alpha }\log \left({\widehat{Y}}_{xyc}\right), if Y_{xyc}\ge {\tau }_1\\ 0, if {\tau }_2<Y_{xyc}<{\tau }_1\\ {\left(1-Y_{xyc}\right)}^{\beta }{\left({\widehat{Y}}_{xyc}\right)}^{\alpha }\log \left(Y_{xyc}-{\widehat{Y}}_{xyc}\right), if Y_{xyc}\le {\tau }_2\end{array},$$

(4)

where $\alpha$ and $\beta$ are the hyperparameters of the focal loss, and N is the sum of the number of positive and negative samples. As shown in Figure 6, the value of the center point of the image is equal to the maximum value 1. The positive samples are set according to the focal loss when $Y_{xyc}\ge {\tau }_1$. The ignored area is defined as${\tau }_2<Y_{xyc}<{\tau }_1$. The model does not calculate the loss for this area and does not perform backpropagation to make the model not pay too much attention to the edge of the bounding box. For negative samples, controllable variable ${\left(1-Y_{xyc}\right)}^{\beta }$ is added. For the negative sample point near the center point, the penalty is relatively large, and the weight of loss ${\left(1-Y_{xyc}\right)}^{\beta }$ is relatively small, which indicates that the distinction between positive and negative at this position is more blurred.

The bounding boxes use CIoU [23] to regress the spatial coordinates. As illustrated in Figure 7, the regression objectives $\tilde{l}$, $\tilde{t}$, $\tilde{r}$, and $\tilde{b}$ are computed for each location on all feature levels. Therefore, the predicted coordinates of the upper left and lower right corner points are represented as $\left({\tilde{x}}_1,{\tilde{y}}_1\right),\left({\tilde{x}}_2,{\tilde{y}}_2\right)$, where ${\tilde{x}}_1={\tilde{p}}_x-\tilde{l}, {\tilde{y}}_1={\tilde{p}}_y-\tilde{t},{\tilde{x}}_2={\tilde{p}}_x+\tilde{r},{\tilde{y}}_2={\tilde{p}}_y+\tilde{b}$. The CIoU resolves the issue of no overlap between the bounding box and ground truth, which results in more stable bounding box regression because it considers the bounding box distance, overlap rate, scale, and penalty terms. Additionally, this may help to avoid divergence throughout the training phase.

The CIoU loss function includes an impact term $\beta v$ based on the DIoU [23], which takes the length-to-width ratio of the predicted and ground truth boxes into account:

$${ℒ}_{reg}=\frac{1}{N_{Pos}}{\displaystyle \sum }_{i\in Pos}^{N}1-Io{U}_i+\frac{{\rho }^2\left(b_i,b_i^{gt}\right)}{c_i^2}+\beta v$$

(5)

$$\beta =\frac{v}{\left(1-Io{U}_i\right)+v}$$

(6)

$$v=\frac{4}{{\pi }^2}{\left(\left(arctan\frac{w_i^{gt}}{h_i^{gt}}\right)-arctan\frac{w_i}{h_i}\right)}^2,$$

(7)

where β is a trade-off parameter and v is a parameter used to determine the aspect ratio consistency. Additionally, ρ() denotes the distance between the center points of the bounding box and ground truth, and c denotes the diagonal length of the smallest enclosing box that encompasses both boxes.

The binary cross-entropy loss is adopted for the aspect-aware head loss. $As{p}_i$ represents the predicted aspect ratio for each detected bounding box, and ${\widehat{Asp}}_i$ is the aspect ratio of the corresponding ground truth box. During training, the gradients from the aspect-aware head loss function can update the weights of regression and the classification branch using backpropagation, which promotes the detection performance of the proposed model. The loss function of the aspect-aware head is defined as

$${ℒ}_{asp}=\frac{1}{N_{Pos}}{{\displaystyle \sum }}_{i\in Pos}^{N}BCE\left(\sigma (As{p}_i\right),\sigma \left({\widehat{Asp}}_i\right)),$$

(8)

where $BCE$ is the binary cross-entropy loss, $\sigma ()$ is the sigmoid function, and $N_{Pos}$ is the number of positive samples. Additionally,

$${ℒ}_{total}={ℒ}_{cls}+{ℒ}_{reg}+{ℒ}_{asp},$$

(9)

is the total loss function obtained by adding three sub-loss functions (${ℒ}_{cls}$: classification head, ${ℒ}_{asp}$: aspect-aware head, and ${ℒ}_{reg}$: regression head). ${ℒ}_{reg}$ and ${ℒ}_{asp}$ are calculated exclusively for positive samples.

3.4. Multi-Scale Detection

As shown in Figure 8, overlapping ground truths may produce uncertainty, where the same point represents the position of different objects, which is difficult to resolve during training. In this paper, a multi-scale prediction approach is demonstrated to solve the problem of semantic diversity. Following the FPN [24] and the pyramid attention network [25], a method for detecting objects of various sizes using different levels of feature layers is proposed. A pyramid is created using a five-scale feature map denoted by the subscripts $\left\{P_3,P_4,P_5,P_6,P_7\right\}$. $P_3,P_4$ and $P_5$ are extracted from $\left\{C_3,C_4,C_5\right\},$ and top-down convolution is performed to mitigate the deterioration caused by increasing the depth of the convolutional layers. $P_6 \mathrm{and} P_7$ are each processed using 3 × 3 convolutions with two strides from $P_5, P_6$. Objects of different scales are mapped on different feature layers for prediction to avoid object occlusion and overlap. Multi-level detection distributes information throughout the many feature layers, which may increase the efficiency of the feature maps and hence detection performance.

4. Experience and Results

4.1. Dataset

Detection performance in traffic scenes was evaluated using the BCTSDB [26] and KITTI [27] datasets. BCTSDB is a traffic sign collection that comprises 15,690 images and 25,243 annotations, with labels classified as prohibitory, required, or warning. The KITTI dataset contains 7481 training images and 7518 test images, with 80,256 labeled objects for three categories (i.e., vehicle, pedestrian, and cyclist).

4.2. Implementation Details

The network structure was constructed using PyTorch, and the default hyperparameters used were the same as those for MMDetection [28] unless otherwise stated. Two NVIDIA TITAN V graphics cards with 24 GB combined video random access memory were used for the configuration. To train the model, stochastic gradient descent was used, with an initial learning rate of $2.5\times {10}^{-2}$ and a warm-up ratio of 0.1. The weight decay and momentum were set to 0.0001 and 0.9, respectively. The backbone network was established using pre-trained weights from ImageNet [29], and other layers used Xavier [30] for parameter initialization. The input images were scaled to a full scale of 640 × 640, maintaining the aspect ratio.

4.3. Validation Experiment

First, the importance of the ignored area is discussed. Experiments verified that the ignored area not only reduced the loss value and increased its stability, but also improved the convergence speed significantly, as illustrated in Figure 9.

The anchor-based methods set too many redundant anchor boxes, which resulted in the number of positive samples being much smaller than the number of negative samples: approximately 1:1000. The anchor-free methods discarded preset anchors and used key points for regression, but the imbalanced problem still existed.

Table 1. Detection results with different sample ratios on BCTSDB.

Model	Sample Ratio	Backbone	AP	AP50	AP75
Faster R-CNN	1:1000	ResNet50 [31]	52.1	81.8	65.2
	1:100		63.5	90.8	77.5
	1:10		68.9	94.2	84.4
	1:3		70.2	94.7	86.0

indicates that the proportion of positive and negative samples affected the detection performance of the model. Unbalanced ratios led to a sharp decrease in the detection performance of the model. As shown in

Table 2. Detection results of different models with different sample definition methods on BCTSDB.

Model	Sample Selection	Backbone	AP	AP50	AP75
YOLOv3	Ori.	ResNet50	59.5	92.7	70.4
	Ours.		64.7	94.2	73.8
RetinaNet	Ori.	ResNet50	59.7	89.4	71.2
	Ours.		63.1	93.7	74.3
FCOS	Ori.	ResNet50	68.6	95.8	83.9
	Ours.		69.1	96.2	84.2

, by replacing the original sample definition methods of different models with the proposed method, the proposed sample definition method alleviated the imbalance problem, thereby improving model detection performance.

The loss function in RetinaNet, CenterNet, and FCOS was replaced with the proposed loss function to test the effectiveness of the proposed loss function. Then the results were compared to those predicted by the original models. As shown in

Table 3. Detection results of different models with different loss functions on BCTSDB.

Model	Backbone	Loss Function	AP	AP50	AP75
RetinaNet	ResNet50	Focal loss	59.7	89.4	71.2
		Aspect aware loss	61.8	91.2	72.6
CenterNet	ResNet50	Gaussian Focal loss	69.4	95.6	85.4
		Aspect aware loss	70.5	96.2	87.1
FCOS	ResNet50	Focal loss	68.6	95.8	83.9
		Aspect aware loss	70.8	97.2	84.6

, the experimental results proved that the proposed loss function provided significant improvements, which indicates that the aspect ratio of the bounding box played a positive role in updating the model weights.

As shown in

Table 4. Effectiveness experiment for the aspect-aware head for different models on BCTSDB.

Model	Backbone	AP	AP50
RetinaNet	ResNet50	59.7	89.4
RetinaNet + Aspect Head		61.2	90.1
FCOS		68.6	95.8
FCOS + Aspect Head		69.2	96.2

, an aspect-aware head was added to different detection models. The experiments demonstrated that it helped the model to learn the aspect ratio distribution of different objects, thus improving the detection accuracy effectively.

A comparison of the performance of the different approaches is summarized in

Table 5. Comparison of results with other methods on the BCTSDB val dataset.

Model	Backbone	AP	AP50	AP75	APS	APM	APL
Faster R-CNN [11]	ResNet50	70.2	94.7	86.0	65.3	76.5	84.5
Cascade R-CNN [12]	ResNet50	75.8	96.7	92.5	72.9	79.3	89.2
SSD [14]	ResNet50	55.4	84.3	67.2	45.2	69.8	81.5
RetinaNet [8]	ResNet50	59.7	89.4	71.2	47.2	72.5	83.3
YOLOv3 [6]	Darknet53	59.5	92.7	70.4	54.2	70.1	83.8
CenterNet [4]	ResNet50	69.4	95.6	85.4	60.1	75.8	87.1
FCOS [5]	ResNet50	68.6	95.8	83.9	62.7	75.7	83.9
AspectNet (Ours.)	ResNet50	76.2	97.3	93.4	72.4	80.7	86.7

, which includes the detection results for the anchor-based and anchor-free methods (CenterNet and FCOS). The anchor-based detectors included multi-stage algorithms and single-stage algorithms. AspectNet achieved high detection accuracy, thereby resulting in more competitive outcomes, with AP AP50 AP75 values of 76.2%, 97.3%, and 93.4%, respectively.

The proposed method was also tested on KITTI. Compared with other existing detectors, AspectNet had much better detection accuracy, as shown in

Table 6. Comparison results for detection methods on the KITTI val dataset.

	Car			Pedestrian			Cyclist
	Easy (%)	Moderate (%)	Hard (%)	Easy (%)	Moderate (%)	Hard (%)	Easy (%)	Moderate (%)	Hard (%)
Regionlets [32]	84.75	76.45	59.70	73.14	61.15	55.21	70.41	58.72	51.83
Faster R-CNN [1]	87.97	79.11	70.62	78.97	65.24	60.09	71.40	61.86	53.97
Sensekitti [33]	94.79	93.17	84.38	82.72	68.41	62.72	82.90	73.48	64.03
Mono3D [34]	84.52	89.37	79.15	80.30	67.29	62.23	77.19	65.15	57.88
MS-CNN [35]	93.87	88.68	76.11	85.71	74.89	68.99	84.88	75.30	65.27
SSD [14]	87.34	87.74	77.27	50.38	48.41	43.46	48.25	52.31	52.13
ASSD [36]	89.28	89.95	82.11	69.07	62.49	60.18	75.23	76.16	72.83
RFBNet [37]	87.31	87.27	84.44	66.16	61.77	58.04	74.89	72.05	71.01
AspectNet (Ours.)	96.16	93.87	88.17	87.15	76.57	68.42	86.15	77.11	71.24

.

The detection results are shown in Figure 10 and Figure 11 on the KITTI and BCTSDB datasets, respectively. The results demonstrate the proposed method’s effectiveness in traffic scenarios.

5. Conclusions

In this paper, an anchor-free-based object detection method in autonomous driving was proposed. The main contribution is that an aspect-aware prediction head is added at the end of the detector. The accuracy of the detection on BCTSDB can be improved by adding aspect-aware prediction head in RetinaNet and FCOS. The improved sample definition method was used to alleviate the problem of sample imbalance; the comparison results on BCTSDB datasets show that the detection accuracy can be improved by replacing the original sample definition method in YOLO V3, FCOS and Retinanet models. The proposed loss function was added to strengthen the learning weight of the center point, and the validation experiment shows that this can improve the detection accuracy. An overall validation on public datasets demonstrated that the proposed method can achieve a significant improvement in detection accuracy. The AP50 and AP75 of the proposed method are 97.3% and 93.4% on BCTSDB, and the average accuracies of car, pedestrian, and cyclist are 92.7%, 77.4% and 78.2% on KITTI, respectively, which indicates that the proposed method achieves better results compared to other methods. The proposed method improved detection accuracy, but it still encountered many challenges when applied to real traffic scenarios. In autonomous driving scenarios, higher accuracy is required, and this can be further improved in future work. The experiment in this paper is trained on public datasets and real traffic scenes facing challenging with complex lighting and weather factors. This issue can be addressed by considering Transformers and domain adaptation in future work.