G-Rep: Gaussian Representation for Arbitrary-Oriented Object Detection

Abstract

Typical representations for arbitrary-oriented object detection tasks include the oriented bounding box (OBB), the quadrilateral bounding box (QBB), and the point set (PointSet). Each representation encounters problems that correspond to its characteristics, such as boundary discontinuity, square-like problems, representation ambiguity, and isolated points, which lead to inaccurate detection. Although many effective strategies have been proposed for various representations, there is still no unified solution. Current detection methods based on Gaussian modeling have demonstrated the possibility of resolving this dilemma; however, they remain limited to OBB. To go further, in this paper, we propose a unified Gaussian representation called G-Rep to construct Gaussian distributions for OBB, QBB, and PointSet, which achieves a unified solution to various representations and problems. Specifically, PointSet- or QBB-based object representations are converted into Gaussian distributions and their parameters are optimized using the maximum likelihood estimation algorithm. Then, three optional Gaussian metrics are explored to optimize the regression loss of the detector because of their excellent parameter optimization mechanisms. Furthermore, we also use Gaussian metrics for sampling to align label assignment and regression loss. Experimental results obtained on several publicly available datasets, such as DOTA, HRSC2016, UCAS-AOD, and ICDAR2015, show the excellent performance of the proposed method for arbitrary-oriented object detection.

1. Introduction

With the development of deep convolutional neural networks (CNNs), object detection, especially arbitrary-orientation object detection [1,2,3,4,5], has developed rapidly and a variety of methods have been proposed. Arbitrary-orientation object detection has a broad range of applications in various scenarios, such as the analysis of aerial images [6] and SAR images [7], scene text detection [8], industry and agriculture automation [9,10,11], facial recognition [12], and target detection in retail scenes [13].

Popular representations of the arbitrary-oriented object are divided into the oriented bounding box (OBB) [14], the quadrilateral bounding box (QBB) [15] and the point set (PointSet) [16]. Each of these representations encounters intrinsic challenges because of the properties of its definitions, which are summarized as follows and illustrated in Figure 1.

• PointSet uses several individual points to represent the overall arbitrary-oriented object. The independent optimization between the points makes the trained detector very sensitive to isolated points, particularly for objects with large aspect ratios, because a slight deviation causes a sharp drop in the intersection-over-union (IoU) value. As shown in Figure 1a, although most of the points are predicted correctly, an outlier makes the final prediction fail. Therefore, the joint optimization loss (e.g., IoU loss [17,18,19]) based on the point set is more popular than the independent optimization loss (e.g., $L_n$ loss).
• As a special case of PointSet, QBB is defined as the four corners of a quadrilateral bounding box. In addition to the inherent problems of PointSet described above, QBB also suffers from the representation ambiguity problem [15]. Quadrilateral detection often sorts the points first (as shown in Figure 1b, represented by the green box) to facilitate point matching between the ground-truth and prediction bounding boxes to calculate the final loss. Although the red prediction box in Figure 1b does not satisfy the sorting rule and obtains a large loss value accordingly using the $L_n$ loss, this prediction is correct according to the IoU-based evaluation metric.
• OBB is the most popular choice for oriented object representation because of its simplicity and intuitiveness. However, the boundary discontinuity and square-like problem are obstacles to high-precision locating, as detailed in [5,20,21,22]. Figure 1c illustrates the boundary problem of OBB representation, considering the OpenCV acute angle definition ($\theta \in [-\pi /2,0)$) as an example [14]. The height (h) and width (w) of the box swap at the angle boundary, resulting in a sudden change in the loss value, which is coupled with the periodicity of the angle and makes regression difficult.

To improve detection performance, many researchers have proposed solutions for some issues, which mainly include the boundary discontinuity [3,20], square-like problem [21,23], representation ambiguity [15], and isolated points. For instance, in previous works [3,20], the authors aimed to solve the boundary discontinuity in OBB. DCL [21] dynamically adjusts the periodicity of loss weights through the aspect ratio to alleviate the square-like problem in OBB. RSDet [23] uses the modulated loss to smooth the boundary loss jump in both OBB and QBB. RIDet [15] uses the Hungarian matching algorithm to eliminate the representation ambiguity caused by the ordering of points in QBB. However, there is still no concise and unified solution to these problems because different solutions require different model settings.

Recent methods, such as GWD [5] and KLD [22], have broken the paradigm of existing regular regression frameworks from a unified regression loss perspective. Specifically, OBB is mapped to the Gaussian distribution using a matrix transformation, as shown in the upper left of Figure 2, and then robust Gaussian regression losses are designed. Although promising results have been achieved, the limitations of the OBB representation make them not truly unified solutions in terms of representation. Additionally, although several distances for the Gaussian distribution have been explored and devised as the regression losses, the metric for dividing positive and negative samples in label assignment (i.e, sample selection) has not been changed accordingly and IoU is still used.

In this paper, we aim to develop a fully unified solution to the problems that result from various representations. Specifically, the PointSet representation and its special cases, QBB, are converted into the Gaussian distribution, and the parameters of the Gaussian distribution are evaluated using the maximum likelihood estimation (MLE) algorithm [24]. Furthermore, three evaluation metrics are explored for computing the similarity of two Gaussian distributions, and Gaussian regression losses based on these metrics are designed. Accordingly, the selection of positive and negative samples is modified using the Gaussian metric by designing fixed and dynamic label assignment strategies. The entire pipeline is shown in Figure 2. The highlights of this paper are as follows:

• To uniformly solve the different problems introduced by different representations (OBB, QBB, and PointSet), Gaussian representation (G-Rep) is proposed to construct the Gaussian distribution using the MLE algorithm.
• To achieve an effective and robust measurement for the Gaussian distribution, three statistical distances, the Kullback–Leibler divergence (KLD) [25], the Bhattacharyya distance (BD) [26], and the Wasserstein distance (WD) [27], are explored and corresponding regression loss functions are designed and analyzed.
• To realize the consistency in measurement between sample selection and loss regression, fixed and dynamic label assignment strategies are constructed based on a Gaussian metric to further boost performance.
• Extensive experiments were conducted on several publicly available datasets, e.g., DOTA, HRSC2016, UCAS-AOD, and ICDAR2015, and the results demonstrated the excellent performance of the proposed techniques for arbitrary-oriented object detection.

2. Related Work

2.1. Oriented Object Representations

The most popular representation of oriented object detection uses the five-parameter OBB, the IoU value as the metric in label assignment, and Smooth $l_1$ as the regression loss to regress the five parameters $(x,y,w,h,\theta )$ [2,3,4,14,20,23], where $(x,y)$, w, h, and $\theta$ denote the center coordinates, width, height, and angle of the box, respectively. Additionally, some researchers have proposed methods using an eight-parameter QBB to represent the object and Smooth $l_1$ as the regression loss to regress the four corner points of QBB [15,28]. The recent anchor-free method CFA [16] uses flexible PointSet (i.e., point set) to represent the oriented objects, inspired by the horizontal object detection method RepPoints [29]. In addition, other complex representations exist, such as polar coordinates [30,31] and middle lines [32]. In previous works, GWD [5] and KLD [22] transform OBB to Gaussian distribution, while GBB (Gaussian bounding box) [33] does not need to obtain a Gaussian representation underlying the OBB and can be seen as an elliptical intrinsic representation of the object. Moreover, GWD [5], KLD [22], and GBB [33] are all anchor-based methods. In this paper, G-Rep applies the Gaussian representation to the anchor-free method for the first time, converting irregularly distributed points into a Gaussian distribution. In addition, we design label assignment strategies based on Gaussian distance metrics. Compared with previous works, G-Rep is less computationally intensive, more accurate in its positioning, and achieves better performance.

2.2. Regression Loss in Arbitrary-Oriented Object Detection

The mainstream regression loss for OBB and QBB is Smooth $l_1$, which encounters boundary discontinuity and square-like problems. To solve these problems, SCRDet [3] and RSDet [23] adopt the IoU–Smooth $l_1$ loss and modulated loss to smooth the boundary loss jump. CSL [20] and DCL [21] transform angular prediction from regression to classification. RIDet [15] uses the representation invariant loss to optimize the bounding box regression. Using the IoU value as the regression loss [17] has become a research topic of great interest in oriented object detection, which can avoid partial problems caused by the regression of the angle parameter or point ordering. Many variants of the IoU loss have subsequently been developed, such as GIoU [18], DIoU [19] and PIoU [13]. However, at the same time, the losses of IoU series suffer from infeasible optimization, slow convergence speed, and gradient vanishing in the case of non-overlapping bounding boxes. GWD [5] and KLD [22] construct the Gaussian distribution for OBB, which overcomes the dilemma of object detection and demonstrates the possibility of a unified solution for various issues. However, the aforementioned two works are limited to OBB representation and are not truly unified solutions in terms of representation.

2.3. Label Assignment Strategies

Label assignment plays a vital role in detection performance, and many fixed and dynamic label assignment strategies have been proposed. Classic object detection methods, such as Faster RCNN (region-based convolutional neural network) [34] and RetinaNet [35], adopt a fixed maximum IoU strategy, which requires predefined thresholds for positive and negative samples in advance. To overcome the difficulty in setting hyper-parameters, ATSS (adaptive training sample selection) [36] uses statistical characteristics to calculate dynamic IoU thresholds. Furthermore, PAA (probabilistic anchor assignment) [37] adaptively separates anchors into positive and negative samples for a ground-truth bounding box in a probabilistic manner. Additionally, other excellent dynamic label assignment strategies exist. For instance, DAL (dynamic anchor learning) [38] dynamically assigns labels according to a defined matching degree, and FreeAnchor [39] dynamically selects labels under the maximum likelihood principle. Nevertheless, these methods still rely on the IoU value as the main indicator to evaluate the quality of the sample.

In addition, some excellent works focus on optimized models to improve detection results. For example, Wang et al. [40] proposed an adaptive feature-aware method, which performs brilliantly in real-time object detection tasks. Wang et al. [41] proposed an absorption pruning method for the object detection network in remote-sensing images. Tong et al. [42] designed a compound semantic feature fusion method to generate an effective semantic description for better pixel-wise object center point interpretation. Ma et al. [43] proposed a feature split–merge–enhancement network (SME-Net) to handle objects with significant scale differences. Yu et al. [44] considered the spatial distribution property of objects to generate higher-quality candidate bounding boxes. Li et al. [45] proposed a meta-learning-based method for few-shot object detection on remote-sensing images. Although the above research efforts have advanced the development of object detection, the study of object representation has not received sufficient attention.

3. Proposed Method

The three main contributions of this paper are as follows: First, Gaussian distribution is constructed for PointSet and QBB representation, which overcomes the dilemma shown in previous studies [5,22] that the Gaussian distribution can only be applied to OBB. Second, new regression loss functions are designed and analyzed for supervising network learning based on the Gaussian distribution. Third, the measurement between label assignment and loss regression based on the Gaussian distribution is aligned, and the new fixed and dynamic label assignment strategies are designed accordingly.

Figure 2 shows an overview of the proposed method. In the training phase, the predicted bounding box and ground truth box are converted into the Gaussian distributions in a manner according to their respective representations. For example, the MLE algorithm is used for the boxes in PointSet or QBB representation [46], and the OBB uses the matrix conversion described in [5], or is converted to QBB and then uses the MLE algorithm. Then, the devised fixed or dynamic G-Rep label assignment strategy is used to select samples based on Gaussian distance metrics (KLD, BD, or WD). Finally, regression loss functions based on the Gaussian distance metrics are designed to minimize the distance between two Gaussian distributions. In the testing phase, the output in OBB form is obtained from the trained model, which is constructed based on the unified Gaussian representation.

3.1. Object Representation Based on Gaussian Distribution

PointSet. RepPoints [29] is an anchor-free method, which is also a baseline method used in this paper. It is constructed with a backbone network, an initial detection head and a refined detection head. The object is represented as a set of adaptive sample points (i.e., PointSet) and the regression framework adopts deformable convolution [47] for point learning. The object is represented as PointSet R, which is defined as:

$$R={\left\{\left(x_i,y_i\right)\right\}}_{i=1}^K,$$

(1)

where $\left(x_i,y_i\right)$ are the coordinates of the i-th point and K is the number of points in a point set, set to nine by default according to [29], which is a trade-off between performance and efficiency. In the refined detection head, the model predicts offset $(\mathsf{\Delta }{x}_i,\mathsf{\Delta }{y}_i)$ for refinement, and the new refined predicted point set can be simply expressed as:

$$R^{\prime }={\left\{\left(x_i+\mathsf{\Delta }{x}_i,y_i+\mathsf{\Delta }{y}_i\right)\right\}}_{i=1}^K.$$

(2)

QBB. The baseline with QBB representation is constructed on the anchor-based method Cas-RetinaNet proposed in [15], which contains a backbone network and two detection heads. QBB is defined as the four corner points of the object ($Q={\left\{\left(x_i^q,y_i^q\right)\right\}}_{i=1}^4$). Note that the four corner points of QBB must be sorted in advance to match the corners of the given ground truth representation one-to-one for regression in the original QBB baseline. Additionally, from the definitions of the three representations, we can deduce that QBB can be regarded as a special case of PointSet, and OBB can be regarded as a special case of QBB. Therefore, constructing the Gaussian distribution for PointSet is extremely generalized, which is also the major focus of this paper.

Transformation between PointSet/QBB and G-Rep. Considering $(x_i,y_i)$ as a two-dimensional (2-D) variable $x_i$, its probability density under the Gaussian distribution $\mathcal{N}(\mu ,\mathsf{\Sigma })$ is defined as

$$\mathcal{N}\left({\mathbf{x}}_i\mid \mu ,\mathsf{\Sigma }\right)=\frac{exp\left(-\frac{1}{2}{\left({\mathbf{x}}_i-\mu \right)}^{\mathrm{T}}{\mathsf{\Sigma }}^{-1}\left({\mathbf{x}}_i-\mu \right)\right)}{2\pi \sqrt{det\left(\mathsf{\Sigma }\right)}}.$$

(3)

The mean value $\mu$ and covariance matrix $\mathsf{\Sigma }$ of the Gaussian distribution are calculated using the maximum likelihood estimation (MLE) algorithm [46].

Figure 3 illustrates the Gaussian learning process based on PointSet. The MLE algorithm evaluates the parameters of the Gaussian distribution for the initialized PointSet. The coordinates of the points are updated according to the gradient feedback of the loss and the Gaussian parameters are updated correspondingly.

Transformation between OBB and G-Rep. In previous studies, such as [5], a 2-D Gaussian distribution for OBB is constructed by a matrix transformation. There are two Gaussian transformation methods adopted in this paper: MLE and matrix transformation. The former can be used for the conversion of all representations (OBB/QBB/PointSet), but is inefficient and inaccurate. The latter is more precise, but only supports OBB. Therefore, when transforming ground truth, matrix transformation is chosen to avoid unnecessary bias.

The transformation from a Gaussian distribution to an OBB is necessary for calculating the mean average precision (mAP) in the testing process. As shown in Figure 2, prediction (QBB or PointSet) is obtained in the testing process. Because network learning is supervised based on the object Gaussian distribution, the prediction of the network output is also distributed in the form of the object Gaussian distribution. Therefore, it is necessary to convert the prediction (PointSet or QBB) into the Gaussian distribution, whose parameters are calculated using the MLE algorithm [46]. To go further, the OBB of the predicted object is obtained from the Gaussian distribution. For a predicted Gaussian distribution where $\mu$ and $\mathsf{\Sigma }$ are known, the parameters of the corresponding OBB $(x,y,w,h,\theta )$ can also be obtained using the singular value decomposition (SVD) through a matrix transformation [5].

3.2. Gaussian Distance Metrics

The keystone of an effective regression loss and label assignment is how to compute the similarity between the predicted Gaussian distribution ${\mathcal{N}}_p\left(x_p\mid {\mu }_p,{\mathsf{\Sigma }}_p\right)$ and the ground-truth Gaussian distribution ${\mathcal{N}}_g\left(x_g\mid {\mu }_g,{\mathsf{\Sigma }}_g\right)$. Next, we will focus on three metrics to calculate the distance between two Gaussian distributions and further analyze their characteristics.

Kullback–Leibler Divergence (KLD) [25]. The KLD between two Gaussian distributions is defined as

$$\begin{array}{cc}\hfill D_K\left({\mathcal{N}}_g,{\mathcal{N}}_p\right)=& \frac{1}{2}(tr\left({\mathsf{\Sigma }}_p^{-1}{\mathsf{\Sigma }}_g\right)+ln\frac{\left|{\mathsf{\Sigma }}_p\right|}{\left|{\mathsf{\Sigma }}_g\right|}-2\hfill \\ \hfill & +{\left({\mu }_p-{\mu }_g\right)}^{\top }{\mathsf{\Sigma }}_p^{-1}\left({\mu }_p-{\mu }_g\right)).\hfill \end{array}$$

(4)

Although KLD is not strictly a mathematical distance because it is asymmetric, it is scale-invariant. According to Equation (4), each item of KLD contains shape parameters $\mathsf{\Sigma }$ and center parameters $\mu$. All parameters form a chain coupling relationship and influence each other, which is very conducive to high-precision detection, as demonstrated in previous work [22].

Bhattacharyya Distance (BD) [26]. The BD between two Gaussian distributions is defined as

$$\begin{array}{cc}\hfill D_B\left({\mathcal{N}}_g,{\mathcal{N}}_p\right)=& \frac{1}{8}{\left({\mu }_g-{\mu }_p\right)}^T{\mathsf{\Sigma }}^{-1}\left({\mu }_g-{\mu }_p\right)\hfill \\ \hfill & +\frac{1}{2}ln\left(\frac{\left|\mathsf{\Sigma }\right|}{\sqrt{\left|{\mathsf{\Sigma }}_g{\mathsf{\Sigma }}_p\right|}}\right),\hfill \end{array}$$

(5)

where $\mathsf{\Sigma }=\frac{1}{2}\left({\mathsf{\Sigma }}_p+{\mathsf{\Sigma }}_g\right)$. Although BD is symmetric, it is not an actual distance either, because it does not satisfy the triangle inequality. Similar to KLD, BD is also scale-invariant.

Wasserstein Distance (WD) [27]. The WD between two Gaussian distributions is defined as

$$\begin{array}{cc}\hfill D_W\left({\mathcal{N}}_g,{\mathcal{N}}_p\right)=& {∥{\mu }_p-{\mu }_q∥}^2+Tr\left({\mathsf{\Sigma }}_p\right)+Tr\left({\mathsf{\Sigma }}_q\right)\hfill \\ \hfill & -2Tr\left({\left({\mathsf{\Sigma }}_p^{1/2}{\mathsf{\Sigma }}_q{\mathsf{\Sigma }}_p^{1/2}\right)}^{1/2}\right).\hfill \end{array}$$

(6)

Different from KLD and BD, WD is an actual mathematical distance, which satisfies the triangle inequality and is symmetric. Note that WD is mainly divided into two parts: the distance between the center points ($x,y$) and the coupling terms about h, w, and $\theta$. Although WD can greatly improve the performance of high-precision rotation detection because of the coupling between parts of the parameters, the independent optimization of the center point slightly shifts the detection result [22]. Note that the above-mentioned Gaussian distance metrics are all in the range $[0,+\infty )$, making them unsuitable for direct use in both regression loss and label assignment strategy. The normalization design for the three metrics is discussed in the following sections.

3.3. Regression Loss Based on Gaussian Metric

The actual value obtained by the distance metric between Gaussian distributions is too large to be the regression loss, which leads to convergence difficulties. Therefore, normalization is necessary so that the Gaussian distance can be used as the regression loss. To find appropriate normalized functions for the regression loss, we first set a general form of the function for KLD, BD and WD, which is defined as

$${\mathcal{L}}_{\mathrm{reg}}=1-\frac{1}{\lambda +f\left(D\left({\mathcal{N}}_g,{\mathcal{N}}_p\right)\right)},$$

(7)

where $\lambda$ is the normalization factor and $f(\ast )$ is the normalization function. The actual $\lambda$ and $f(\ast )$ are chosen according to the experiments described in Section 4.2. $D\left({\mathcal{N}}_g,{\mathcal{N}}_p\right)$ refers to the three Gaussian metrics, which are defined in Equations (4)–(6), respectively.

Recall that the most common regression loss for object detection is the Smooth $l_1$ loss. The Smooth $l_1$ loss independently minimizes each parameter of the prediction. Taking the OBB as an example, the center point $(x,y)$, the width w, the height h, and the angle $\theta$ are optimized separately. In fact, different objects have different sensitivities to different parameters. For instance, objects with large aspect ratios are sensitive to angle error. Hence, the decoupled regression approach has difficulty handling various objects with diverse shapes. For different representations, the Smooth $l_1$ loss suffers from different issues, as illustrated in Figure 1. By contrast, all three Gaussian metric-based regression losses avoid issues of boundary discontinuity, square-like problems, representation ambiguity, and isolated points. Among them, WD is strictly the actual distance because of its triangle equality and symmetry, whereas KLD is asymmetric and BD satisfies the triangle inequality. Although the three proposed regression loss functions have different characteristics, the overall difference in detection performance is slight and all are superior to the traditional Smooth $l_1$ or IoU loss.

3.4. Label Assignment Based on Gaussian Metric

Label assignment strategy is another key task for object detection. The most popular label assignment strategy is the IoU-based strategy, which assigns a label by comparing IoU values (proposals and ground truth) with the IoU threshold. However, if the regression loss is based on a Gaussian distribution, inconsistencies can occur when the metrics of label assignment and regression loss are different. Therefore, new label assignment strategies based on the Gaussian distribution are devised. To go further, dynamic label assignment strategies with better adaptability and fewer hyper-parameters are also designed for G-Rep.

Fixed G-Rep Label Assignment. The range of the IoU value is $\left[0,1\right]$ according to the definition of IoU, and the threshold values are selected empirically in the range $\left[0.3,0.7\right]$. However, this strategy is clearly not applicable to the Gaussian distribution distance calculated by the three metrics described in Section 3.2, whose value ranges are not closed intervals. Along with the concept of G-Rep regression loss design, normalized functions for each distance evaluation metric are adopted. The general form of the normalized metric for KLD, BD, and WD used in the label assignment process is defined as

$${\mathcal{S}}_{\mathrm{la}}=\frac{1}{\alpha +{\left[D\left({\mathcal{N}}_g,{\mathcal{N}}_p\right)\right]}^{\beta }},$$

(8)

where $\mathcal{S}$ denotes the normalized metric for evaluating the similarity between the Gaussian distribution of a proposal and the ground truth, and $\alpha$ and $\beta$ are normalization factors whose values are selected based on the experiments described in Section 4.2. The optimal hyper-parameters require empirical and experimental optimization. A series of experiments were conducted with a unified positive threshold of 0.4 and a negative threshold of 0.3, respectively. The results are listed in Table 4, where rows 2–4 demonstrate that optimal hyper-parameter thresholds are difficult to set for different distance metrics; hence, dynamic label assignment strategies based on a Gaussian are further explored.

Dynamic G-Rep Label Assignment. Dynamic G-Rep label assignment strategies are devised based on the three distance metrics in Section 3.2 to avoid the difficulty of selecting the optimal hyper-parameters. Inspired by ATSS [36], the threshold for selecting positive and negative samples is calculated dynamically according to the statistical characteristics of all the normalized distances (calculated in Equation (8)). For the i-th ground truth, the dynamic threshold $\mathcal{T}$ is calculated as

$$\begin{array}{c}\hfill {\mathcal{T}}_i=\frac{1}{J}\sum _{j=1}^J{\mathcal{I}}_{i,j}+\sqrt{\frac{1}{J}\sum _{j=1}^J{({\mathcal{I}}_{i,j}-\frac{1}{J}\sum _{j=1}^J{\mathcal{I}}_{i,j})}^2},\end{array}$$

(9)

where J is the number of candidate samples, and ${\mathcal{I}}_{i,j}$ is the normalized value of KLD, BD, or WD between the i-th ground truth and the j-th proposal, which is calculated in Equation (8). Next, positive samples are selected using the general assignment strategy; that is, candidates are selected whose similarity values are greater than or equal to the threshold ${\mathcal{T}}_i$.

Furthermore, motivated by PAA [37], another approach using Gaussian distributions in the label assignment process is adopted in this paper. Specifically, each sample is assigned a score for quality evaluation. The Gaussian mixture model (GMM) with two components is adopted to model the score distribution of samples, and the parameters of the GMM are calculated using the expectation-maximization algorithm [24]. Finally, samples are assigned labels according to their probabilities. PAA and ATSS are modified and combined to construct the robust dynamic label assignment strategy PATSS in the proposed method.

4. Experiments

4.1. Datasets and Implementation Details

Experiments were conducted on the public aerial image datasets DOTA [48], HRSC2016 [49], UCAS-AOD [50], and scene text dataset ICDAR2015 [51] to verify the superiority of the proposed method.

DOTA [48] is a public benchmark dataset for oriented object detection in aerial images, which contains 15 object categories: plane (PL), baseball diamond (BD), bridge (BR), ground track field (GTF), small vehicle (SV), large vehicle (LV), ship (SH), tennis court (TC), basketball court (BC), storage tank (ST), soccer ball field (SBF), roundabout (RA), harbor (HA), swimming pool (SP), and helicopter (HC). In the experiments, DOTA’s official training and validation dataset sets were selected as the training set, whose images were split into blocks of 1024 × 1024 pixels with an overlap of 200 pixels and scaled to 1333 × 1024 for training.

HRSC2016 [49] is a public aerial image dataset for ship detection. The number of images in the training, validation, and testing sets are 436, 181 and 444, respectively.

The public aerial image dataset UCAS-AOD [50] is another multi-class oriented object detection dataset, and contains two categories: car and airplane.

ICDAR2015 [51] is commonly used for oriented scene text detection and spotting. This dataset contains 1000 training images and 500 testing images.

Experiments were conducted using the MMDetection framework [52] (under the Apache-2.0 License) on 2 Titan V GPUs with 11GB memory. Two baselines of RepPoints with PointSet and Cas-RetinaNet with QBB were defined.

The optimizers of the RepPoints and Cas-RetinaNet frameworks were SGD with a 0.01 initial learning rate and Adam with a 0.0001 initial learning rate, respectively.

The framework for the RepPoints baseline was trained for 12 and 36 epochs on DOTA and HRSC2016 for the ablation experiments, and 40, 80, 120 and 240 epochs on DOTA, HRSC2016, UCAS-AOD and ICDAR2015 with data augmentation, respectively. The framework for the Cas-RetinaNet baseline was trained for 12, 100, 36 and 100 epochs on DOTA, HRSC2016, UCAS-AOD and ICDAR2015, respectively.

The mAP was adopted as an evaluation metric for testing results on DOTA, HRSC2016 and UCAS-AOD. Precision, recall, and F-measure (i.e., F1) were adopted on ICDAR2015, following official criteria.

4.2. Normalized Function Design

The design principle of the normalized factors for regression loss and label assignment is mapping the calculated values of the Gaussian statistical distance to the values in a suitable range. Experiments were conducted to explore the selection of normalization factors and functions mentioned in Equations (7) and (8), the results of which are listed in

Table 1. Experiment results of normalized function design for label assignment ($\mathcal{S}$) and regression loss ($\mathcal{L}$) on HRSC2016.

Metric	Func. of $\mathcal{S}$	Range of $\mathcal{S}$	Func. of $\mathcal{L}$	Range of $\mathcal{L}$	mAP (%)
KLD	$\frac{1}{2+D_K}$	$(0,0.5]$	$1-\frac{1}{0+exp\left(\sqrt{D_K}\right)}$	$[0,1)$	87.32
	$\frac{1}{2+D_K}$	$(0,0.5]$	$1-\frac{1}{0+exp\left(D_K^2\right)}$	$[0,1)$	50.73
	$\frac{1}{2+D_K}$	$(0,0.5]$	$1-\frac{1}{2+\sqrt{D_K}}$	$[0.5,1)$	88.06
	$\frac{1}{1+{\left(\sqrt{D_K}\right)}^3}$	$(0,1]$	$1-\frac{1}{2+\sqrt{D_K}}$	$[0.5,1)$	87.96
BD	$\frac{1}{1+D_B^2}$	$(0,1]$	$1-\frac{1}{1+D_B^2}$	$[0,1)$	81.02
	$\frac{1}{1+D_B^2}$	$(0,1]$	$1-\frac{1}{1+\sqrt{D_B}}$	$[0,1)$	69.32
	$\frac{1}{1+D_B^2}$	$(0,1]$	$1-\frac{1}{1+D_B}$	$[0,1)$	85.32
	$\frac{1}{1+D_B}$	$(0,1]$	$1-\frac{1}{1+D_B}$	$[0,1)$	85.12
WD	$\frac{1}{2+D_W}$	$(0,0.5]$	$1-\frac{1}{2+\sqrt{D_W}}$	$[0.5,1)$	87.04
	$\frac{1}{2+D_W}$	$(0,0.5]$	$1-\frac{1}{0+exp\left(\sqrt{D_W}\right)}$	$[0,1)$	88.24
	$\frac{1}{2+D_W}$	$(0,0.5]$	$1-\frac{1}{1+log(1+D_W)}$	$[0,1)$	88.56
	$\frac{1}{2+\sqrt{D_W}}$	$(0,0.5]$	$1-\frac{1}{1+log(1+D_W)}$	$[0,1)$	87.54

. For the regression loss function, $\lambda$ was chosen from $\{0,1,2\}$, and the normalization function $f(\ast )$ was selected from several frequently-used functions, such as $log(\ast )$, $e^{*}$ and ${\ast }^a$. For label assignment strategy, $\alpha$ was selected from $\{1,2\}$, and $\beta$ was selected from $\{0.5,1,1.5,2\}$. The results in Table 1 show that the specific design of the normalized functions had a significant impact on the detection accuracy. Compared with exponential functions, logarithmic functions and power functions are more suitable for regression loss design. Note that the three Gaussian metrics have different mathematical properties, so that we applied different normalization factors and functions on different metrics in our experiments. Finally, the appropriate normalized functions were selected according to the best results of the experiments (the rows indicated by the bold “mAP” values in Table 1).

4.3. Ablation Study

Table 2. Ablation study of G-Rep for PointSet on DOTA and HRSC2016. $\mathcal{S}$ and $\mathcal{L}$ represent the label assignment strategy and regression loss function, respectively.

Dataset	Rep.	$\mathcal{S}$	$\mathcal{L}$	mAP (%)
DOTA	PointSet	IoU (Max)	GIoU	63.97
	G-Rep	IoU (Max)	${\mathcal{L}}_{\mathrm{KLD}}$	64.63 (+0.66)
		${\mathcal{S}}_{\mathrm{KLD}}$ (Max)	${\mathcal{L}}_{\mathrm{KLD}}$	65.07 (+1.10)
	PointSet	IoU (ATSS)	GIoU	68.88
	G-Rep	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	70.45 (+1.57)
		${\mathcal{S}}_{\mathrm{KLD}}$ (PATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	72.08 (+3.20)
HRSC2016	PointSet	IoU (ATSS)	GIoU	78.07
	G-Rep	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	88.06 (+9.99)
		${\mathcal{S}}_{\mathrm{KLD}}$ (PATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	89.15 (+11.07)

compares the performance of the G-Rep and PointSet on the DOTA dataset. “IoU (Max)” and “IoU (ATSS)” represent the fixed predefined threshold strategy and dynamic ATSS [36] label assignment strategy based on the IoU metric, respectively. The baseline method of PointSet is RepPoints [29] with ResNet50-FPN [53,54]. Note that the IoU for the Gaussian distribution was calculated as the IoU between the box transformed by the Gaussian distribution and ground truth. The predefined positive and negative thresholds in the fixed strategy for IoU and ${\mathcal{S}}_{\mathrm{KLD}}$ were 0.4 and 0.3, respectively. The superiority of G-Rep was reflected in two ways: regression loss and label assignment.

Analysis of regression loss based on G-Rep. Even if only the GIoU was replaced by ${\mathcal{L}}_{\mathrm{KLD}}$, the performance of G-Rep was better than that of PointSet (64.63% vs. 63.97%). The dynamic label assignment strategies avoid the influence of unsuitable hyper-parameters for a fair comparison of the GIoU and Gaussian regression loss. Additionally, the superiority of G-Rep was clearly demonstrated when the dynamic label assignment strategies were used. ${\mathcal{L}}_{\mathrm{KLD}}$ still surpassed the GIoU with the same dynamic label assignment strategy ATSS on DOTA (70.45% vs. 68.88%).

Analysis of label assignment based on G-Rep. The label assignment strategy is another important factor for high detection performance. For the ${\mathcal{L}}_{\mathrm{KLD}}$ loss, Table 2 shows the detection results of the different label assignment strategies. Using KLD resulted in better performance than using IoU as a metric of the label assignment, which demonstrates the effectiveness of aligning the label assignment and regression loss metrics. The optimal fixed negative and positive thresholds for selecting samples are difficult to select, whereas dynamic label assignment strategies avoid this issue. PATSS denotes the combination of the ATSS [36] and PAA [37] strategies. The mAP further reached 70.45% and 72.08% under the more robust dynamic selection strategies ATSS and PATSS, respectively. Without additional features, the combination of the dynamic label assignment strategy and regression loss increased the mAP by 8.11% compared with the baseline method.

Analysis of the advantages for an object with a large aspect ratio. Outliers often cause more serious location errors for objects with large aspect ratios than for square objects.

Table 3. Performance comparison of PointSet and G-Rep for large aspect ratio objects. The ratio number in the parentheses next to the category name in the first row is the mean aspect ratio (ratio of the long side to the short side) of all targets in that category.

Rep.	BR (2.93)	SV (1.72)	LV (3.45)	SH (2.40)	HC (2.34)	mAP (%)
PointSet	46.87	77.10	71.65	83.71	32.93	62.45
G-Rep	50.82	79.33	75.07	87.32	50.63	68.63
	(+3.95)	(+2.23)	(+3.51)	(+3.61)	(+17.70)	(+6.18)

shows that G-Rep was more effective than PointSet for the objects with a large aspect ratio, where the mAP increased by 6.18% for the five typical categories with narrow objects on DOTA because G-Rep was not sensitive to isolated points.

Comparison of different Gaussian distance metrics.Table 4 compares the performances when different evolution metrics, KLD, WD and BD, were used in fixed and dynamic label assignment strategies and regression loss. The performances based on fixed label assignment strategies varied greatly as a result of the hand-crafted hyper-parameters. Therefore, experiments based on dynamic label assignment strategies were constructed to objectively compare the performances of the metrics. The experimental results demonstrate that the overall performance of the G-Rep loss functions surpassed that of the GIoU loss. There were tolerable performance differences between BD and the other two losses, and a slight difference (within 0.5%) between KLD and WD. To further explore whether KLD and WD are more suitable as the regression loss than BD, the label assignment metrics were unified as KLD (rows 5, 8 and 9) for the ablation study of the loss functions. In fact, all three G-Rep losses outperformed the baseline (RepPoints) [29]. There were slight differences between them in detection performance.

Table 4. Comparison of the three Gaussian distances as metrics for label assignment and regression loss on HRSC2016.

Rep.	$\mathcal{S}$	$\mathcal{L}$	mAP (%)
PointSet	IoU (ATSS)	GIoU	78.07
G-Rep	${\mathcal{S}}_{\mathrm{KLD}}$ (Max)	${\mathcal{L}}_{\mathrm{KLD}}$	73.44
	${\mathcal{S}}_{\mathrm{BD}}$ (Max)	${\mathcal{L}}_{\mathrm{BD}}$	46.71
	${\mathcal{S}}_{\mathrm{WD}}$ (Max)	${\mathcal{L}}_{\mathrm{WD}}$	84.39
	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	88.06
	${\mathcal{S}}_{\mathrm{BD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{BD}}$	85.32
	${\mathcal{S}}_{\mathrm{WD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{WD}}$	88.56
	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{BD}}$	88.90
	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{WD}}$	88.80
	${\mathcal{S}}_{\mathrm{BD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	85.32
	${\mathcal{S}}_{\mathrm{BD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{WD}}$	85.28

Table 4. Comparison of the three Gaussian distances as metrics for label assignment and regression loss on HRSC2016.

Rep.	$\mathcal{S}$	$\mathcal{L}$	mAP (%)
PointSet	IoU (ATSS)	GIoU	78.07
G-Rep	${\mathcal{S}}_{\mathrm{KLD}}$ (Max)	${\mathcal{L}}_{\mathrm{KLD}}$	73.44
	${\mathcal{S}}_{\mathrm{BD}}$ (Max)	${\mathcal{L}}_{\mathrm{BD}}$	46.71
	${\mathcal{S}}_{\mathrm{WD}}$ (Max)	${\mathcal{L}}_{\mathrm{WD}}$	84.39
	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	88.06
	${\mathcal{S}}_{\mathrm{BD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{BD}}$	85.32
	${\mathcal{S}}_{\mathrm{WD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{WD}}$	88.56
	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{BD}}$	88.90
	${\mathcal{S}}_{\mathrm{KLD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{WD}}$	88.80
	${\mathcal{S}}_{\mathrm{BD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{KLD}}$	85.32
	${\mathcal{S}}_{\mathrm{BD}}$ (ATSS)	${\mathcal{L}}_{\mathrm{WD}}$	85.28

Ablation study on various datasets.

Table 5. Ablation study of G-Rep for QBB representations on various datasets. The regression loss of G-Rep is the ${\mathcal{L}}_{\mathrm{KLD}}$. “*” denotes that dynamic ATSS-based strategies are adopted.

Dataset	Rep.	Eval.	Gain ↑
DOTA	PointSet *	68.88	–
	G-Rep * (PointSet)	70.45	+1.57
	QBB	63.05	–
	G-Rep (QBB)	67.92	+4.87
HRSC2016	PointSet *	78.07	–
	G-Rep * (PointSet)	88.06	+9.99
	QBB	87.70	–
	G-Rep (QBB)	88.01	+0.31
UCAS-AOD	PointSet *	90.15	–
	G-Rep * (PointSet)	90.20	+0.05
	QBB	88.50	–
	G-Rep (QBB)	88.82	+0.32
ICDAR2015	PointSet *	76.20	–
	G-Rep * (PointSet)	81.30	+5.10
	QBB	75.10	–
	G-Rep (QBB)	75.83	+0.73

shows the experimental results of G-Rep using two baselines on various datasets. The QBB baseline adopted the anchor-based method Cas-RetinaNet [15] (i.e., the cascaded RetinaNet [35]). G-Rep resulted in varying degrees of improvement on the anchor-based baseline with QBB and the anchor-free baseline with PointSet on various datasets.

There are three key aspects to these results:

• Elongated objects. On the datasets containing a large number of elongated objects (e.g., HRSC2016, ICDAR2015), the improvement in G-Rep applied to PointSet was more pronounced than that applied to QBB, as shown in Table 3, mainly because the greater the number of points, the more accurate the Gaussian distribution obtained, and, thus, the more accurate the representation of the elongated object.
• Size of dataset. The performance on the small datasets (e.g., UCAS-AOD) tended to be saturated, so the improvement was relatively small.
• High baseline. Models with a high-performance baseline were hard to improve significantly (e.g., HRSC2016-QBB, DOTA-PointSet).

4.4. Time Cost Analysis

Table 6. Comparison of the time cost on DOTA.

Method	mAP (%)	Params	Speed
RepPoints	70.39	36.1M	24.0fps
S${}^2$ANet	74.12	37.3M	19.9fps
G-Rep (ours)	75.56	36.1M	19.3fps

compares the time cost of G-Rep and the other two methods. Since their parameters are the same, the main difference during inference lies in the post-processing. Although MLE slowed down the inference speed, G-Rep achieved a performance improvement of 5.17% over baseline, which was worthwhile. In addition, G-Rep achieved better performance with fewer parameters based on RepPoints (anchor-free) compared to S${}^2$ANet (anchor-based).

4.5. Comparison with Other Methods

The performance of the proposed method was compared with that of other state-of-the-art detection methods on the DOTA dataset, which is a benchmark aerial image dataset for multi-class oriented object detection. The experimental results are shown in

Table 7. Comparison of various detectors of $\mathrm{mAP}$ values on the OBB-based task of the DOTA-v1.0. “MS” indicates multi-scale training.

Method	Backbone	MS	PL	BD	BR	GTF	SV	LV	SH	TC	BC	ST	SBF	RA	HA	SP	HC	mAP (%)
two-stage:
ICN [1]	R-101	✓	81.40	74.30	47.70	70.30	64.90	67.80	70.00	90.80	79.10	78.20	53.60	62.90	67.00	64.20	50.20	68.20
GSDet [59]	R-101		81.12	76.78	40.78	75.89	64.50	58.37	74.21	89.92	79.40	78.83	64.54	63.67	66.04	58.01	52.13	68.28
RADet [60]	RX-101	✓	79.45	76.99	48.05	65.83	65.45	74.40	68.86	89.70	78.14	74.97	49.92	64.63	66.14	71.58	62.16	69.06
RoI-Transformer [2]	R-101	✓	88.64	78.52	43.44	75.92	68.81	73.68	83.59	90.74	77.27	81.46	58.39	53.54	62.83	58.93	47.67	69.56
CAD-Net [61]	R-101		87.80	82.40	49.40	73.50	71.10	63.50	76.70	90.90	79.20	73.30	48.40	60.90	62.00	67.00	62.20	69.90
SCRDet [3]	R-101	✓	89.98	80.65	52.09	68.36	68.36	60.32	72.41	90.85	87.94	86.86	65.02	66.68	66.25	68.24	65.21	72.61
SARD [62]	R-101		89.93	84.11	54.19	72.04	68.41	61.18	66.00	90.82	87.79	86.59	65.65	64.04	66.68	68.84	68.03	72.95
FADet [63]	R-101	✓	90.21	79.58	45.49	76.41	73.18	68.27	79.56	90.83	83.40	84.64	53.40	65.42	74.17	69.69	64.86	73.28
MFIAR-Net[64]	R-152	✓	89.62	84.03	52.41	70.30	70.13	67.64	77.81	90.85	85.40	86.22	63.21	64.14	68.31	70.21	62.11	73.49
Gliding Vertex [28]	R-101		89.64	85.00	52.26	77.34	73.01	73.14	86.82	90.74	79.02	86.81	59.55	70.91	72.94	70.86	57.32	75.02
CenterMap [65]	R-101	✓	89.83	84.41	54.60	70.25	77.66	78.32	87.19	90.66	84.89	85.27	56.46	69.23	74.13	71.56	66.06	76.03
CSL (FPN-based) [20]	R-152	✓	90.25	85.53	54.64	75.31	70.44	73.51	77.62	90.84	86.15	86.69	69.60	68.04	73.83	71.10	68.93	76.17
RSDet [23]	R-152	✓	89.93	84.45	53.77	74.35	71.52	78.31	78.12	91.14	87.35	86.93	65.64	65.17	75.35	79.74	63.31	76.34
OPLD [66]	R-101	✓	89.37	85.82	54.10	79.58	75.00	75.13	86.92	90.88	86.42	86.62	62.46	68.41	73.98	68.11	63.69	76.43
SCRDet++ [67]	R-101	✓	90.05	84.39	55.44	73.99	77.54	71.11	86.05	90.67	87.32	87.08	69.62	68.90	73.74	71.29	65.08	76.81
one-stage:
P−RSDet [30]	R-101		89.02	73.65	47.33	72.03	70.58	73.71	72.76	90.82	80.12	81.32	59.45	57.87	60.79	65.21	52.59	69.82
${\mathrm{O}}^2-\mathrm{Det}$ [32]	H-104		89.31	82.14	47.33	61.21	71.32	74.03	78.62	90.76	82.23	81.36	60.93	60.17	58.21	66.98	61.03	71.04
ACE [68]	DAL34[69]		89.50	76.30	45.10	60.00	77.80	77.10	86.50	90.80	79.50	85.70	47.00	59.40	65.70	71.70	63.90	71.70
${\mathrm{R}}^3\mathrm{Det}$ [4]	R-152	✓	89.24	80.81	51.11	65.62	70.67	76.03	78.32	90.83	84.89	84.42	65.10	57.18	68.10	68.98	60.88	72.81
BBAVectors [70]	R-101	✓	88.35	79.96	50.69	62.18	78.43	78.98	87.94	90.85	83.58	84.35	54.13	60.24	65.22	64.28	55.70	73.32
DRN [71]	H-104	✓	89.71	82.34	47.22	64.10	76.22	74.43	85.84	90.57	86.18	84.89	57.65	61.93	69.30	69.63	58.48	73.23
GWD [5]	R-152		88.88	80.47	52.94	63.85	76.95	70.28	83.56	88.54	83.51	84.94	61.24	65.13	65.45	71.69	73.90	74.09
RO${}^3$D [72]	R-101	✓	88.69	79.41	52.26	65.51	74.72	80.83	87.42	90.77	84.31	83.36	62.64	58.14	66.95	72.32	69.34	74.44
CFA [16]	R-101		89.26	81.72	51.81	67.17	79.99	78.25	84.46	90.77	83.40	85.54	54.86	67.75	73.04	70.24	64.96	75.05
KLD [22]	R-50		88.91	83.71	50.10	68.75	78.20	76.05	84.58	89.41	86.15	85.28	63.15	60.90	75.06	71.51	67.45	75.28
${\mathrm{S}}^2\mathrm{A}-\mathrm{Net}$ [6]	R-101		88.70	81.41	54.28	59.75	78.04	80.54	88.04	90.69	84.75	86.22	65.03	65.81	76.16	73.37	58.86	76.11
PolarDet [31]	R-101	✓	89.65	87.07	48.14	70.97	78.53	80.34	87.45	90.76	85.63	86.87	61.64	70.32	71.92	73.09	67.15	76.64
DAL (${\mathrm{S}}^2\mathrm{A}-\mathrm{Net}$) [38]	R-50	✓	89.69	83.11	55.03	71.00	78.30	81.90	88.46	90.89	84.97	87.46	64.41	65.65	76.86	72.09	64.35	76.95
GGHL [73]	D-53		89.74	85.63	44.50	77.48	76.72	80.45	86.16	90.83	88.18	86.25	67.07	69.40	73.38	68.45	70.14	76.95
DCL (${\mathrm{R}}^3\mathrm{Det}$) [21]	R-152	✓	89.26	83.60	53.54	72.76	79.04	82.56	87.31	90.67	86.59	86.98	67.49	66.88	73.29	70.56	69.99	77.37
RIDet [15]	R-50		89.31	80.77	54.07	76.38	79.81	81.99	89.13	90.72	83.58	87.22	64.42	67.56	78.08	79.17	62.07	77.62
QBB (baseline)	R-50		77.52	57.38	37.20	65.97	56.29	69.99	70.04	90.31	81.14	55.34	57.98	49.88	56.01	62.32	58.37	63.05
PointSet (baseline)	R-50		87.48	82.53	45.07	65.16	78.12	58.72	75.44	90.78	82.54	85.98	60.77	67.68	60.93	70.36	44.41	70.39
G-Rep (QBB)	R-101		88.89	74.62	43.92	70.24	67.26	67.26	79.80	90.87	84.46	78.47	54.59	62.60	66.67	67.98	52.16	70.59
G-Rep (PointSet)	R-50		87.76	81.29	52.64	70.53	80.34	80.56	87.47	90.74	82.91	85.01	61.48	68.51	67.53	73.02	63.54	75.56
G-Rep (PointSet)	RX-101	✓	88.98	79.21	57.57	74.35	81.30	85.23	88.30	90.69	85.38	85.25	63.65	68.82	77.87	78.76	71.74	78.47
G-Rep (PointSet)	Swin-T	✓	88.15	81.64	61.30	79.50	80.94	85.68	88.37	90.90	85.47	87.77	71.01	67.42	77.19	81.23	75.83	80.16

, where R-50/101, RX-101, D-53, H-104 and Swin-T denote ResNet-50/101 [53], ResNeXt-101 [55], DarkNet53 [56], Hourglass-104 [57], and Swin-Transformer [58], respectively.

The results in Table 7 show that the one-stage methods exhibited a trend of outperforming the two-stage methods, and the G-Rep based on the anchor-free method RepPoints achieved the state-of-the-art performance. With the excellent backbone Swin-T [58] and some common tricks, such as data augmentation, G-Rep achieved 80.16% mAP on the DOTA dataset. In addition, the results showed that G-Rep performed better on some object categories with large aspect ratios (e.g., BR, LV, and HC). Based on the anchor-free baseline, G-Rep was superior to the anchor-free method ACE [68] 8.64% mAP. G-Rep is a pioneering new paradigm in oriented object detection.

To further verify the effectiveness of the proposed method and compare it with other state-of-the-art detectors, a series of experiments were conducted on HRSC2016 and UCAS-AOD. The results are shown in

Table 8. Comparison of the mAP of various rotation methods on HRSC2016.

Method	mAP (%)
RoI-Transformer [2]	86.20
RSDet [23]	86.50
Gliding Vertex [28]	88.20
BBAVectors [70]	88.60
${\mathrm{R}}^3\mathrm{Det}$ [4]	89.26
DCL [21]	89.46
G-Rep (QBB)	88.02
G-Rep (PointSet)	89.46

and

Table 9. Comparison of the AP with state-of-the-art methods on UCAS-AOD.

Method	Car	Airplane	mAP (%)
RetinaNet [35]	84.64	90.51	87.57
Faster RCNN [34]	86.87	89.86	88.36
RoI-Transformer [2]	88.02	90.02	89.02
RIDet-Q [15]	88.50	89.96	89.23
RIDet-O [15]	88.88	90.35	89.62
DAL [38]	89.25	90.49	89.87
G-Rep (QBB)	87.35	90.30	88.82
G-Rep (PointSet)	89.64	90.67	90.16

, respectively. The results demonstrate that the models using G-Rep achieved state-of-the-art performance. Although DCL [21] achieved the same performance as G-Rep (PointSet) on the ship detection dataset HRSC2016, G-Rep did not require preset anchors and required fewer hyper-parameters and computations, proving the robustness of G-Rep. Table 9 shows that G-Rep (PointSet) achieved 0.93% and 0.54% higher mAP than RIDet (OBB) and RIDet (QBB) [15], respectively, indicating the superiority of using Gaussian representation for objects.

To further demonstrate the generalizability of G-Rep, we also conducted some extended experiments on the oriented text detection dataset ICDAR2015. The experimental results are listed in

Table 10. Comparison of the performance of different methods on ICDAR2015.

Method	Precision	Recall	F-Measure
GWD [5]	80.5	74.0	77.1
RRPN [74]	82.2	73.2	77.4
SCRDet [3]	81.3	78.9	80.1
RO${}^3$D [72]	83.9	78.6	81.2
G-Rep (QBB)	80.5	71.4	75.8
G-Rep (PointSet)	81.6	81.1	81.3

. Precision, recall and the F-measure were used as the evaluation metrics following the official criteria of the ICDAR2015 dataset. The experimental results showed that G-Rep (PointSet) outperformed the one-stage method RO${}^3$D [72] and the two-stage method SCRDet [3] on recall and the F-measure, demonstrating the universality and superior performance of G-Rep.

4.6. Visualization Analysis

Figure 4 visualizes the detection results of PointSet and G-Rep on HRSC2016. The points of PointSet were distributed at the boundary of objects, while the points of the G-Rep were distributed in the interior of the object. Therefore, G-Rep is superior in terms of accurate localization and is not sensitive to outliers.

More visualization examples of the experimental results on DOTA and UCAS-AOD datasets are shown in Figure 5 and Figure 6. The visualization results show that points of G-Rep were concentrated in the interior of the object, which is a relatively accurate localization of the object. The points of PointSet, on the other hand, focused on the boundaries of the object, which can easily cause deviations in localization. These visualization results of different kinds of objects on different datasets are sufficient to demonstrate the superiority of G-Rep.

4.7. More Discussion

Although the proposed method provides a uniform representation of the Gaussian distribution for various representations of the input, an obvious limitation is that the format of the output can only be OBB. The output points are dispersed as the Gaussian distribution inside the object, and OBB is transformed from the Gaussian distribution of the output points. Additionally, the angle prediction of the square-like object transformed by the isotropic Gaussian is inaccurate. Note that the irregular PointSet and QBB representations are more accurate than the OBB representation for most objects. However, for some square-like regular objects, OBB representation is more accurate than PointSet and QBB representations.

5. Conclusions

The main contribution of this study is that G-Rep was proposed to construct the Gaussian distribution on PointSet and QBB, which overcame the limitation of Gaussian applications for current object detection methods and achieved a truly unified solution. Additionally, various label assignment strategies for the Gaussian distribution were designed, and the metrics were aligned between label assignment and regression loss. Through extensive experiments, G-Rep was demonstrated to be effective in many areas, including:

• G-Rep uses Gaussian representation to alleviate the challenges posed by other common representations. The experimental results in Table 5 show that G-Rep resulted in a substantial increase of up to 9.99% of mAP on the HRSC2016 dataset when applied to PointSet.
• G-Rep uses the normalized Gaussian distance for the regression loss function and a label assignment strategy instead of the IoU-based metric, which resulted in significant increases in mAP, up to 3.20% on the DOTA dataset and 11.07% on the HRSC2016 dataset, as shown in Table 2.
• G-Rep utilizes a Gaussian distribution to guide the regression of points in PointSet and QBB, which makes the detection results less sensitive to outliers and more accurate for elongated objects. As shown in Table 3, G-Rep resulted in a 6.18% improvement in mAP for elongated objects on the DOTA dataset.

More importantly, G-Rep resolves the current dilemma and provides inspiration for exploring other forms of label assignment strategies and regression loss functions. However, despite the effectiveness of G-Rep, additional matrix transformations resulting from Gaussian representation may slow down the inference speed, and the loss of directional information caused by the isotropic Gaussian distribution can slightly reduce the detection accuracy for square-like objects. The above limitations will be investigated further in future work.