Next Article in Journal
Robust Pulse Rate Measurements from Facial Videos in Diverse Environments
Next Article in Special Issue
Passive Radar Tracking in Clutter Using Range and Range-Rate Measurements
Previous Article in Journal
Deep Multi-Scale Features Fusion for Effective Violence Detection and Control Charts Visualization
Previous Article in Special Issue
Meter Wave Polarization-Sensitive Array Radar for Height Measurement Based on MUSIC Algorithm
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multi-Target Tracking Algorithm Combined with High-Precision Map

1
Beijing Rxbit Electronic Technology Co., Ltd., Beijing 100081, China
2
School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
3
School of Physics & Electronic Engineering, Hubei University of Arts and Science, Xiangyang 441053, China
4
The Trade Desk Inc., 42 N Chestnut St., Ventura, CA 93001, USA
5
School of Software Engineering, Tongji University, Shanghai 201804, China
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Sensors 2022, 22(23), 9371; https://doi.org/10.3390/s22239371
Submission received: 27 October 2022 / Revised: 24 November 2022 / Accepted: 28 November 2022 / Published: 1 December 2022
(This article belongs to the Special Issue Radar Sensors for Target Tracking and Localization)

Abstract

:
On high-speed roads, there are certain blind areas within the radar coverage, especially when the blind zone is in curved road sections; because the radar does not have the measurement point information in multiple frames, it is easy to have a large deviation between the real trajectory and the filtered trajectory. In this paper, we propose a track prediction method combined with a high-precision map to solve the problem of scattered tracks when the targets are in the blind area. First, the lane centerline is fitted to the upstream and downstream lane edges obtained from the high-precision map in certain steps, and the off-north angle at each fitted point is obtained. Secondly, the normal trajectory is predicted according to the conventional method; for the extrapolated trajectory, the northerly angle of the lane centerline at the current position of the trajectory is obtained, the current coordinate system is converted from the north-east-up (ENU) coordinate system to the vehicle coordinate system, and the lateral velocity of the target is set to zero in the vehicle coordinate system to reduce the error caused by the lateral velocity drag of the target. Finally, the normal trajectory is updated and corrected, and the normal and extrapolated trajectories are managed and reported. The experimental results show that the accuracy and convergence effect of the proposed methods are much better than the traditional methods.

1. Introduction

In recent years, the number of cars has been growing, and the pressure on highway operations has risen. The key point of building an intelligent traffic system (ITS) is monitoring road traffic. The most common sensor used for highway monitoring is a vision sensor [1], such as a camera commonly found on highways. Although video image processing technology is a new traffic detection method that has been gradually developed in recent years, it is wireless, flexible in use, can detect multiple traffic parameters at once, and has a large detection range, and it has a very broad application prospect with the rise of high-definition cameras, deep learning, artificial intelligence, and other technologies [2,3,4,5,6,7,8,9,10]. However, the detection performance of vision sensors is significantly reduced under bad weather conditions such as low light, rain, snow, and fog. In addition, vehicle detection on highways using vision sensors is also prone to driver privacy issues. Other researchers have proposed using satellite remote sensing images to identify vehicles and roads to control traffic congestion on roads and identify traffic density quickly and accurately [11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27]. However, this method is computationally intensive and is not conducive to real-time processing. In the development of highways into the process of intelligent construction, millimeter-wave radar has the features of all-day and all-weather operation, is independent of the external light environment, and the radar sensor has the characteristics of high accuracy, small size, and being a good estimate of the location of the vehicle target and speed information, so it has been widely used in the field of high-speed traffic [28].
In recent years, more and more scholars have begun researching the use of radar to monitor the traffic environment, and the problem of multiple object tracking (MOT) [29,30,31,32,33] has been a subject of great importance to scholars both at home and abroad. Different approaches for blind zone tracking have been proposed. A Joint Integrated Probabilistic Data Association Filter (JIPDAF) [34] method is proposed to track pedestrians in front vision and vehicle blind spots. The JIPDAF method is based on a camera and radar sensor, and it is easier to suppress false alarms of blind spot radar (BSR). Researchers have proposed a non-myopic multi-sensor collaborative scheduling method [35] for tracking the ground maneuvering target in the presence of the detection blind zone (DBZ). The method not only considers the immediate reward of the sensor scheduling action, but also takes into account the future expected reward over the prediction time horizon. Most of the existing methods of the blind zone for target detection and tracking are based on various sensors, but few researchers are involved in target tracking in a single radar blind zone on the highway.
There are certain blind zones in the radar coverage, especially when there are radar blind spots on curved roads. In addition, due to the absence of measurement point information in multiple frames, it is easy to have a significant deviation between the real track and the filtered track, resulting in a scattered track. Therefore, to solve this problem, this article proposes a track prediction method combined with high-precision maps. First, the lane centerline is fitted to the upstream and downstream lane edges obtained from the high-precision map in certain steps, and the off-north angle at each fitted point is obtained. Secondly, the normal trajectory is predicted according to the conventional method, and for the extrapolated trajectory, the northerly angle of the lane centerline at the current position of the trajectory is obtained, the current coordinate system is converted from the north-east-up (ENU) coordinate system to the vehicle coordinate system, and the lateral velocity of the target is set to zero in the vehicle coordinate system to reduce the error caused by the lateral velocity drag of the target. Finally, the normal trajectory is updated and corrected, and the normal and extrapolated trajectories are managed and reported.
This paper is organized as follows: the relevant theory is briefly introduced in Section 2. Section 3 provides an overview of the algorithmic system, including the main techniques of the algorithm and the algorithmic process. Experimental results and analysis are given in Section 4. Finally, the conclusions are given in Section 5.

2. Relevant Theory

2.1. Coordinate System Conversion

In the article, four coordinate systems are mainly involved: WGS-84 geodetic coordinate system, north-east-up (ENU) coordinate system, radar coordinate system, and vehicle coordinate system.
The WGS-84 coordinate system is the abbreviation of the 1984 World Geodetic System (WGS). It represents points on the Earth in terms of three quantities: latitude, longitude, and altitude, also known as the LLA coordinate system. As shown in Figure 1, latitude is the angle between the normal of the reference ellipsoidal plane across point P and the equatorial plane, with latitude values ranging from −90° to +90°, positive for the northern hemisphere and negative for the southern hemisphere. Longitude is the angle between the meridian plane passing through point P and the prime meridian, with longitude values between −180° and +180°. The altitude is the distance normal to the reference ellipsoid from point P. It is negative inside the reference ellipsoid and positive outside.
The Earth-Centered, Earth-Fixed (ECEF) coordinate system is a Cartesian coordinate system with the center of the earth as the origin, as shown in Figure 2. The z-axis is parallel to the Earth’s axis and points to the north pole, the x-axis points to the intersection of the prime meridian and the equator, and the y-axis points to the intersection of 90° east longitude and the equator. Where a represents the major axis of the earth and b represents the minor axis of the earth.
The local Cartesian coordinate system is also called the north-east-up (ENU or NEU) coordinate system, with the selected observation point as the origin, the due east direction as the x-axis, the due north direction as the y-axis, and the direction pointing to the sky as the z-axis.
The radar coordinate system is a Cartesian coordinate system with the radar installation position as the origin, with the radar beam pointing direction as the y-axis, and with the y-axis oriented 90° clockwise as the x-axis.
The vehicle coordinate system is a Cartesian coordinate system with the selected observation point as the origin, with the direction of vehicle travel as the y-axis, and with the y-axis 90° clockwise as the x-axis.

2.1.1. LLA Coordinate System Converted to ENU Coordinate System

The conversion of the LLA coordinate system to the ENU coordinate system requires the use of the ECEF coordinate system as a transition. For a point in space, the conversion relationship from longitude–latitude–altitude coordinates ( L , B , H ) to the Cartesian coordinate system ( X , Y , Z ) is as follows.
{ X = ( N + H ) cos B cos L Y = ( N + H ) cos B sin L Z = [ N ( 1 e 2 ) + H ] sin B
where N = a / 1 e 2 sin 2 B and N is the radius of curvature in the prime vertical at that point. e 2 = ( a 2 b 2 ) / a 2 ; a , b , and e are the long semi-axis, short semi-axis, and the first eccentricity of the reference ellipsoid corresponding to the ECEF coordinate system, respectively.
The conversion of the ECEF coordinate system ( X , Y , Z ) to the ENU coordinate system ( E , N , U ) is calculated as follows.
{ E = sin L 0 ( X X 0 ) + cos L 0 ( Y Y 0 ) N = sin B 0 cos L 0 ( X X 0 ) sin L 0 sin B 0 ( Y Y 0 ) + cos B 0 ( Z Z 0 ) U = cos B 0 cos L 0 ( X X 0 ) + cos B 0 sin L 0 ( Y Y 0 ) + sin B 0 ( Z Z 0 )
where ( X 0 , Y 0 , Z 0 ) is the representation of the coordinate origin of the ENU coordinate system in the ECEF coordinate system, and ( L 0 , B 0 , H 0 ) is the representation of the coordinate origin of the ENU coordinate system in the LLA coordinate system.
Equation (2) can also be expressed as
[ E N U ] = R [ X X 0 Y Y 0 Z Z 0 ]
where R = [ sin L 0 cos L 0 0 sin B 0 cos L 0 sin B 0 sin L 0 cos B 0 cos B 0 cos L 0 cos B 0 sin L 0 sin B 0 ] .

2.1.2. LLA Coordinate System Converted to Radar Coordinate System

First, the LLA coordinate system is converted into the ENU coordinate system, the installation position where the current radar is located is used as the origin of the ENU coordinate system, and then the ENU coordinate system is rotated by an angle around the z-axis according to the magnitude of the off-north angle of the radar (the angle between the radar irradiation direction and the due north direction) to obtain the radar coordinate system. The conversion of the ENU coordinate system to the radar coordinate system is performed as follows.
Figure 3 shows a schematic diagram of the conversion of the ENU coordinate system to the radar coordinate system, where the conversion formula is
{ a x 1 = a x cos ψ + b y sin ψ b y 1 = b y cos ψ a x sin ψ

2.1.3. LLA Coordinate System Converted to Vehicle Coordinate System

The process of converting the LLA coordinate system to the vehicle coordinate system is the same as the process of converting the LLA coordinate system to the radar coordinate system, except that the origin of the vehicle coordinate system is the radar installation position with the number 0.

2.2. System Model

2.2.1. Equation of State

On high-speed roads, vehicles generally maintain a uniform motion, so we use the constant-velocity (CV) model to describe the motion of vehicle targets. The equation of state is an assumption of the law of motion of the target. For example, assuming that the target moves in a uniform linear motion in a two-dimensional plane, the state ( x k , y k ) of the target at moment t k under the discrete-time system may be expressed as
x k = x 0 + v t k = x 0 + v x k T
y k = y 0 + v y t k = y 0 + v y k T
where ( x 0 , y 0 ) is the position of the target at the initial moment, v x and v y are the velocities of the target in the x-axis and y-axis, respectively, and T is the sampling time interval.
Equations (5) and (6) can be expressed in recursive form as
x k + 1 = x k + v x T = x k + x ˙ k T
y k + 1 = y k + v y T = y k + y ˙ k T
Consider the impossibility of obtaining an accurate model of the target and many unpredictable phenomena. In other words, the target cannot do absolute uniform motion, and its velocity must have some small random fluctuations; for example, the target in the process of uniform speed, the driver or environmental disturbances, etc., can cause unpredictable changes in velocity, and these small changes in velocity can be regarded as process noise to the model. Thus, after the introduction of process noise, Equations (7) and (8) should be expressed as follows
x k + 1 = x k + x ˙ k T + 1 2 v x T 2
y k + 1 = y k + y ˙ k T + 1 2 v y T 2
where v x and v y denote the random variation in the target x-axis and y-axis velocities, respectively, and the velocity of the target can be expressed as
x ˙ k + 1 = x ˙ k + v x T
y ˙ k + 1 = y ˙ k + v y T
In the uniform velocity model, the state vector describing the dynamic characteristics of the system is X ( k ) = [ x k   y k   x ˙ k   y ˙ k ] T . Then, Equations (9)–(12) can be expressed in matrix form as
[ x ( k + 1 ) y ( k + 1 ) x ˙ ( k + 1 ) y ˙ ( k + 1 ) ] = [ 1 0 T 0 0 1 0 T 0 0 1 0 0 0 0 1 ] [ x ( k ) y ( k ) x ˙ ( k ) y ˙ ( k ) ] + [ 0.5 T 2 0 0 0.5 T 2 T 0 0 T ] [ v x v y ]
That is, the target state equation can be expressed as
X ( k + 1 ) = F ( k ) X ( k ) + Γ ( k ) v ( k ) = F ( k ) X ( k ) + V ( k )
where v ( k ) = [ v x , v y ] T is the process noise vector, F ( k ) = [ 1 0 T 0 0 1 0 T 0 0 1 0 0 0 0 1 ] is the state transfer matrix of the system, Γ ( k ) = [ 0.5 T 2 0 0 0.5 T 2 T 0 0 T ] is the process noise distribution matrix, and V ( k ) is a zero-mean, white Gaussian measurement noise sequence with covariance Q ( k ) .

2.2.2. Measurement Equation

The measurement equation is an assumption of the radar measurement process; for linear systems, the measurement equation can be expressed as
Z ( k + 1 ) = H ( k + 1 ) X ( k + 1 ) + W ( k + 1 )
where Z ( k + 1 ) is the measurement vector; H ( k + 1 ) is the measurement matrix; W ( k + 1 ) is a zero-mean, white Gaussian measurement noise sequence with covariance R ( k + 1 ) , i.e., the measurement noise at different moments is independent of each other, and the process noise sequence is assumed to be independent of the measurement noise sequence and the initial state of the target.
When modeling the target in the two-dimensional plane with uniform motion, the corresponding state variable is X ( k ) = [ x k   y k   x ˙ k   y ˙ k ] T , the measurement vector is Z ( k ) = [ x k   y k   x ˙ k   y ˙ k ] T , and the corresponding measurement matrix is
H ( k ) = [ 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ]

2.3. Filtering Model

The Kalman filter is a filter under the linear mean square error criterion for non-smooth problems with finite observation intervals.
The minimum mean square error estimate for the time-varying case can be defined as
X ^ ( k | k ) = E [ X ( k ) | Z k ]
where
Z k = { Z ( j ) , j = 1 , 2 , , k }
The state error covariance matrix accompanying Equation (17) is defined as
P ( k | k ) = E { [ X ( k ) X ^ ( k | k ) ] [ X ( k ) X ^ ( k | k ) ] T | Z k }
Applying the expectation operator conditioned on Z k to Equation (14) yields a one-step prediction of the state as
X ^ ( k + 1 | k ) = E [ X ( k + 1 ) | Z k ] = E [ F ( k ) X ( k ) + V ( k ) | Z k ] = F ( k ) X ^ ( k | k )
The error of the predicted value is
X ˜ ( k + 1 | k ) = X ( k + 1 ) X ^ ( k + 1 | k ) = F ( k ) X ˜ ( k | k ) + V ( k )
The one-step prediction covariance is
P ( k + 1 | k ) = E [ X ˜ ( k + 1 | k ) X ˜ T ( k + 1 | k ) | Z k ] = E { [ F ( k ) X ˜ ( k | k ) + V ( k ) ] [ F T ( k ) X ˜ T ( k | k ) + V T ( k ) ] | Z k } = F ( k ) P ( k | k ) F T ( k ) + Q ( k )
Similarly, the predicted value of the measurement can be expressed as
Z ^ ( k + 1 | k ) = E [ Z ( k + 1 ) | Z k ] = E { [ H ( k + 1 ) X ( k + 1 ) + W ( k + 1 ) ] | Z k } = H ( k + 1 ) X ^ ( k + 1 | k )
In turn, the difference between the predicted and measured values of the measurement can be found as
Z ˜ ( k + 1 | k ) = Z ( k + 1 ) Z ^ ( k + 1 | k ) = H ( k + 1 ) X ˜ ( k + 1 | k ) + W ( k + 1 )
The predicted covariance (or new interest covariance) of the measure is
S ( k + 1 ) = E [ Z ˜ ( k + 1 | k ) Z ˜ T ( k + 1 | k ) | Z k ] = E { [ H ( k + 1 ) X ˜ ( k + 1 | k ) + W ( k + 1 ) ] [ H T ( k + 1 ) X ˜ T ( k + 1 | k ) + W T ( k + 1 ) ] | Z k } = H ( k + 1 ) P ( k + 1 | k ) H T ( k + 1 ) + R ( k + 1 )
The Kalman gain is
K ( k + 1 ) = P ( k + 1 | k ) H T ( k + 1 ) S 1 ( k + 1 )
Further, the state update equation at moment k + 1 can be found as
X ^ ( k + 1 | k + 1 ) = X ^ ( k + 1 | k ) + K ( k + 1 ) v ( k + 1 )
where v ( k + 1 ) is the measurement residual, i.e.,
v ( k + 1 ) = Z ˜ ( k + 1 | k ) = Z ( k + 1 ) Z ^ ( k + 1 | k )
The covariance update equation is
P ( k + 1 | k + 1 ) = P ( k + 1 | k ) P ( k + 1 | k ) H T ( k + 1 ) S 1 ( k + 1 ) H ( k + 1 ) P ( k + 1 | k ) = [ I K ( k + 1 ) H ( k + 1 ) ] P ( k + 1 | k )

3. System Overview

3.1. Main Technologies

3.1.1. High-Precision Map Lane Line Fitting

In the radar Cartesian coordinate system, two lane centerlines are fitted according to the Y values of the road edges given by the high-precision map in certain steps (e.g., 10 m), but the Y values are discrete and the X values corresponding to the Y values selected by a certain step distance each time do not necessarily exist in the high-precision map, so we need to interpolate to obtain the corresponding X values. The Lagrangian interpolation method can find a polynomial that takes exactly the observed value at each observed point, and the details of the algorithm are as follows.
For a certain polynomial function, it is known that there are k + 1 given taking points: ( x 0 , y 0 ) , ( x 1 , y 1 ) , , ( x k , y k ) , where x j ( j = 0 , 1 , 2 , , k ) corresponds to the position of the independent variable, and y j ( j = 0 , 1 , 2 , , k ) corresponds to the value of the function taken at this position.
Assuming that any two distinct x j are different from each other, the Lagrangian interpolation polynomial obtained by applying the Lagrangian interpolation formula is
L ( x ) = j = 0 k y j l j ( x )
where each l j ( x ) is a Lagrangian fundamental polynomial (or interpolating basis function) with the expressions
l j ( x ) = i = 0 , i j k x x i x j x i = x x 0 x j x 0 x x j 1 x j x j 1 x x j + 1 x j x j + 1 x x k x j x k
The Lagrangian fundamental polynomial l j ( x ) is characterized by taking the value 1 on x j and 0 at other points x i ( i j ) .
In the actual Lagrangian interpolation process, we use k = 2 and select three actual high-precision map lane information points.

3.1.2. Data Association Algorithm

In the data association process, we use the computationally simple Nearest-Neighbor Data Association (NNDA) Algorithm, which works by first setting up a tracking wave gate, and the echoes obtained by the initial screening of the tracking wave gate become candidate echoes.
A tracking wave gate is a subinterval in the tracking space centered at the predicted position of the tracked target. The size of the tracking wave gate should be designed to ensure that the correct echoes are received with a certain probability. The simplest method of correlation wave gate formation is to define a rectangular region in the tracking space, i.e., a rectangular wave gate.
Let the measurement residual v ( k + 1 ) , the i-th component of the measure Z ( k + 1 ) , and the predicted value of the measure Z ^ ( k + 1 | k ) be denoted by v i ( k + 1 ) , Z i ( k + 1 ) , and Z ^ i ( k + 1 | k ) , respectively, and the i-th row and j-th column element of the predicted covariance S ( k + 1 ) be denoted by S i j . When all components of the measure Z ( k + 1 ) satisfy Equation (32), the measure Z ( k + 1 ) is said to fall into the rectangular wave gate and the measure is a candidate echo.
| v i ( k + 1 ) | = | Z i ( k + 1 ) Z ^ i ( k + 1 | k ) | K G S i i i = 1 , 2 , 3 , 4
where K G is the wave gate constant.
In practice, we first set a smaller wave gate ( K G = 2 ) and initially select the association list of the track and the measurement; the association list includes the index of the track and the measurement points; the correlation weights and the association weights are calculated as shown in Equation (33); the square of the Mahalanobis distance between the track and the measurement points is used as the weights.
The remaining unassociated tracks are associated with the measurement points for a second large wave gate ( K G = 5 ), and to prevent the large wave gate association weights from being smaller than the association weights of the small wave gate, the association weights of the large wave gate are all increased by a larger fixed value, and the associated results are merged into the association list. Finally, the optimal association is obtained from the association list to obtain the final association list according to the idea of the greedy algorithm.
d 2 ( Z ) = [ Z Z ^ ( k + 1 | k ) ] T S 1 ( k + 1 ) [ Z Z ^ ( k + 1 | k ) ]

3.1.3. Track Prediction Combined with High-Precision Map

When the vehicle target is located in the radar blind zone, continuous multi-frame prediction of the track tracking process is needed as the track filtering result when there is no measurement point, especially when the blind zone is at a curve. The filtering result without measurement is likely to make the filtered track deviate greatly from the real track, resulting in the track divergence. Therefore, instead of using the velocity vector of the track itself for position prediction within the radar blind spot, the coordinates of the track position and velocity information are converted from the ENU coordinate system to the vehicle coordinate system by matching the off-north angle of the lane centerline. In the vehicle coordinate system, the lateral velocity of the target becomes zero, and the target is predicted according to the lane direction, which can prevent the divergence of the track due to its lateral velocity component. After completing the track filtering, the vehicle coordinate system is transferred back to the ENU coordinate system.

3.2. Tracking Master Process

On the highway, the tracking of vehicle targets requires the measurement information of the target to correct the predicted state of the target. However, when the target is in the radar blind zone, especially in the curved road, as shown in Figure 4, the predicted track with no measurement point correction in multiple frames will be output as the final track, which is likely to cause the output track to deviate from the real track or even diverge. To solve this problem, when the target is driving in a blind zone or the measurement is missed, the track can be predicted by combining high-precision maps. The algorithm steps are as follows:
Step 1: Divide the four edges of the high-precision map according to the coverage area of radar power.
Step 2: Using the radar’s latitude and longitude coordinates and off-north angle information given by the high-precision map, the four sidelines within each radar coverage area are converted from the LLA coordinate system to the corresponding radar coordinate system under the corresponding radar.
Step 3: Using the road edge data in the radar coordinate system, according to the method in Section 3.1.1, the road edge is fitted with a certain step (such as 10 m) to obtain the corresponding horizontal coordinate values, and then the coordinates of the road centerline at the corresponding distance are obtained.
Step 4: By taking the position relationship between the centerline points of two adjacent lanes, the offset angle Δ θ of each differential point in the radar coordinate system is obtained, and then, combined with the current radar off-north angle θ , an angle θ = θ + Δ θ can be calculated as the off-north angle of the lane of the differential section.
Step 5: Traverse the list of tracks reported at the last moment in the ENU coordinate system, and predict the position and speed information of each track.
Step 6: Adaptively set the predictive model noise Q for the transient track list according to the variable t i c k values and update the covariance matrix P of the track according to Equation (22).
Parameter t i c k is mainly used to set the track reporting module and the covariance matrix Q of noise. In the experiment, when t i c k > 8 , the track is considered to be a stable track, reaching the standard of track reporting. For the covariance matrix Q of noise, set Q = k d i a g [ 0.0009   0.0009   0.0004   0.0004 ] during initialization, where k is the coefficient.
When t i c k 8 , set k = 1 , and it is assumed that the mobility of the target movement is small, which is beneficial to filtering the unstable track. When t i c k > 8 , setting k = 100 can effectively track the target with strong mobility. The settings of the above parameters are all obtained after the comparative analysis of the experimental results, and they have obvious advantages in the experimental results.
Step 7: Associate the trajectory with the measurement points once with a smaller wave gate ( K G = 2 ) to obtain a list of all associations, which includes the indexes and association weights of the tracks and the measurement points.
Step 8: Associate the remaining unassociated tracks with the measurement points for a second time with a large wave gate, and merge the association results into the association list.
Step 9: Obtain the optimal association from the association list to obtain the final association list according to the greedy algorithm.
Step 10: Update the track states according to Equations (23)–(29).
Step 11: For the track without measuring point information, the prediction result of Step 5 is canceled, the track prediction method combined with a high-precision map is adopted, and add 1 to the track quality parameter a g e . First, the track is transferred from the ENU coordinate system to the radar coordinate system according to Equation (4), and then the target track is transferred from the ENU coordinate system to the vehicle coordinate system according to the Y value of the track matching the off-north angle of the corresponding lane center line. The lateral velocity component of the target is set to zero, and the predicted track is predicted according to the vehicle motion model. Finally, the predicted track is converted back to the ENU coordinate system.
Parameter a g e is mainly used for the track termination module and track extrapolation judgment. When a g e > 600 , the track termination condition is met. When 5 a g e 600 , that is, there are more than 5 frames of the track that are not related to the measurement point, which meets the conditions of track extrapolation. The track filtering algorithm combined with the high-precision map proposed in this paper is adopted. When a g e > 600 is satisfied or there are measurement points on the track for more than 5 consecutive frames, stop track extrapolation and set a g e to zero.
The track quality parameter a g e and t i c k are two criteria to measure the track quality in track management, which are related to whether the target track can be related to the measurement points. If there are no measurement points in the current track, the a g e value will be increased by 1, and if there are measurement points in the track, the t i c k value will be increased by 1.
Step 12: Initial track generation for measurement points that are not associated.
Step 13: For the updated list of transient tracks, report the tracks that meet the reporting conditions.
Figure 5 shows the overall flow of the tracking algorithm in this paper. Therefore, when the target is within the radar irradiation range, the traditional track filtering method is used to filter the target track. When the target is in the radar blind area, the multi-target tracking algorithm combined with the high-precision map proposed in this paper is used to filter the target. Compared with the traditional multi-target tracking algorithm, we consider the situation of target tracking in the radar blind zone.

4. Results

This section aims to evaluate the track filtering results of the proposed method for targets in radar blind zones in complex scenarios. We compare the filtering results of a radar deployed on a highway in two scenarios: on a straight road and a curved road.

4.1. Parameters

The traffic radar used in this experiment is based on a 79 GHz millimeter-wave radar, cascaded with a long-range antenna, a medium-range antenna, and a short-range antenna, each with a different downward tilting installation angle, to achieve a long-range radar coverage of 45 m~550 m, breaking the limitation of the traditional antenna beam coverage and achieving a longer-range radar coverage to meet the current road detection requirements. The specific parameters of the traffic radar are shown in Table 1.
The radar is mounted in the middle of the green belt between the upstream lane and downstream lane at a height of 10 m. Figure 6 depicts the coverage of traffic radar, from which it can be seen that there is a 45 m radar blind zone for a single radar. On high-speed roads, as in Figure 7, two radars are mounted at one radar installation point with opposite irradiation directions, so there is a radar blind zone of approximately 90 m. The supporting components have no shielding surface, which will not interfere with the electromagnetic waves emitted by radar, and will not affect the detection of objects.

4.2. Straight Roads

In a straight road, the radar covers five upstream lanes and five downstream lanes. Figure 8 represents the filtered output of the track, which shows that there is a blind zone at the location of the radar installation.
Figure 9 shows the radar track filtering results. Figure 9a is the track filtering result of the traditional method. It can be seen from the figure that although there is no track deviation of the vehicle target in the radar blind zone, the track filtering at the junction of the radar blind zone and the radar irradiation area is not smooth, so it can be concluded that the track filtering result obtained by the traditional method in the radar blind zone is quite different from the real track. Compared with the track filtering results in Figure 9a, the track filtering results in Figure 9b combined with high-precision maps are smoother, and the track filtering results in the radar blind area are closer to the real track of the target. There are some false detection targets in lane 4 and lane 5 of the downstream lane in Figure 9, which belong to the signal detection part. However, this paper discusses the multi-target tracking processing, which belongs to the data processing part, and does not deal with the false detection generated by the signal processing part.
In the experiment, the data we used for the test contained thousands of frames of radar echo data, so what we presented on the result chart was only a small part of the test data. In Figure 9b, the five lanes above the radar are upstream lanes, and the downstream lanes of the five lanes below the radar have opposite driving directions. The red track of the fourth upstream lane and the blue track of the fourth downstream lane are the beginning of the data we selected, but the track of the target before that has not been selected by us. The blue track of the fourth upstream lane and the red track of the third downstream lane are the last stages of our data selection, after which the target track has not been selected. Therefore, the discontinuous tracks in the figure are not the same target, only because some target tracks are not completely presented in the process of intercepting data.

4.3. Curved Roads

In the curved road, the radar covers five upstream lanes and five downstream lanes. Figure 10 represents the filtering results of the output trajectories. Track termination due to lack of measurement point information in the blind zone and the extent of the radar’s blind zone can be seen in the figure.
Figure 11 shows the radar trajectory filtering results. Figure 11a shows the results of the conventional track filtering, from which it can be seen that there are two lanes in the radar blind zone where the trajectories deviate from the current lane, and once the target is out of the radar blind zone, the trajectories that deviate from the current lane continue to be filtered out normally due to the availability of measurement point information to correct for the predicted trajectories. If the target trajectories deviate too much from the lane in the radar blind zone, the trajectories will likely not be correlated with the measurement points information at the junction of the radar blind zone and the radar illuminated zone, resulting in tracks termination and restart. Figure 11b shows the results of the trajectory filtering in radar blind zones combined with high-precision maps. It can be seen from the figure that the trajectory prediction method combined with the high-precision map in the radar blind zone can solve the problem of the trajectory deviating from the current lane very well so that even if there is no measurement point information in multiple frames in the radar blind zone, the trajectory can be filtered out well, and there is no problem where the trajectory and the measurement point are not associated at the junction of the radar blind zone and the radar illumination area.

5. Discussion

This paper describes a tracking algorithm combined with a high-precision map in detail. It solves the problem that the track filtering result deviates from the lane in the blind area of radar because there is no measurement point information in multiple frames. The smoothness of the track at the junction of the radar blind area and the radar irradiation area is improved.
In this paper, the multi-target tracking algorithm combined with a high-precision map has been verified in two scenes: a straight road and a curved road. The two selected radars are installed in opposite directions, and the longest distance detected by a single radar can reach 550 m. The blind zone of the two radars is about 90 m, and each radar covers five upstream lanes and five downstream lanes.
In the straight road scene, the road in the radar blind zone is straight, and the driving direction of the target is unchanged. Therefore, in the straight road scene, it is rare that the target track deviates from the lane, but the track connection at the junction of the radar blind zone and the radar irradiation area may not be smooth. Compared with the traditional algorithm, the tracking algorithm proposed in this paper can make the track smoother in a straight road scene.
In the curved road scene, the driving direction of the target will change with the road direction. However, there is no measurement information in the radar blind area, so the traditional tracking filtering method cannot track the target effectively, and it is easy for the target track to deviate from the current lane line. Compared with the traditional algorithm, the tracking algorithm proposed in this paper can reduce the drag effect on the lateral distance of the target by changing the coordinate system in the course of track prediction. Therefore, the target track can be accurately tracked in the curved road scene, the situation that the track deviates from the lane line is reduced, and the track is smoother.
Compared with previous studies, the novelty of this paper lies in establishing a complete set of target-tracking algorithms in traffic scenes. Considering the problem of tracking the target in the blind area of radar, the target is tracked and filtered by combining the method of a high-precision map. Compared with the existing methods, this method can effectively track the targets in more complex road scenes and improve the smoothness of the filtered track.

Author Contributions

Conceptualization, Q.A. and Y.C.; methodology, J.Z.; software, J.Z.; validation, Q.A., Y.C., J.Z., and S.W.; formal analysis, S.W.; writing—original draft preparation, Y.C.; writing—review and editing, J.Z., S.W., and F.H.; supervision, J.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded in part by the High Technology Research Project of Xiangyang Science and Technology Bureau under grant number 2020ABH001273, and in part by the Fundamental Research Funds for the Central Universities under grant number 2022-5-YB-01. National Natural Science Foundation of China under Grant 61936014, in part by the Shanghai Sailing Program under Grant 21YF1450100, and in part by the Fundamental Research Funds for the Central Universities under Grant 2022-5-YB-01.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Maurin, B.; Masoud, O.; Papanikolopoulos, N.P. Tracking all traffic: Computer vision algorithms for monitoring vehicles, individuals, and crowds. IEEE Robot. Autom. Mag. 2005, 12, 29–36. [Google Scholar] [CrossRef]
  2. Robert, K. Video-based traffic monitoring at day and night vehicle features detection tracking. In Proceedings of the 2009 12th International IEEE Conference on Intelligent Transportation Systems, St. Louis, MO, USA, 4–7 October 2009; pp. 1–6. [Google Scholar] [CrossRef]
  3. Saunier, N.; Sayed, T.; Lim, C. Probabilistic Collision Prediction for Vision-Based Automated Road Safety Analysis. In Proceedings of the 2007 IEEE Intelligent Transportation Systems Conference, Bellevue, WA, USA, 30 September–3 October 2007; pp. 872–878. [Google Scholar] [CrossRef] [Green Version]
  4. Saunier, N.; Sayed, T. A feature-based tracking algorithm for vehicles in intersections. In Proceedings of the 3rd Canadian Conference on Computer and Robot Vision (CRV’06), Quebec, QC, Canada, 7–9 June 2006; p. 59. [Google Scholar] [CrossRef] [Green Version]
  5. Rostamianfar, O.; Janabi-Sharifi, F.; Hassanzadeh, I. Visual Tracking System for Dense Traffic Intersections. In Proceedings of the 2006 Canadian Conference on Electrical and Computer Engineering, Ottawa, ON, Canada, 7–10 May 2006; pp. 2000–2004. [Google Scholar] [CrossRef]
  6. Alhuthali, S.A.H.; Zia, M.Y.I.; Rashid, M. A Simplified Traffic Flow Monitoring System Using Computer Vision Techniques. In Proceedings of the 2022 2nd International Conference on Computing and Information Technology (ICCIT), Tabuk, Saudi Arabia, 25–27 January 2022; pp. 167–170. [Google Scholar] [CrossRef]
  7. Kaushik, S.; Raman, A.; Rao, K.V.S.R. Leveraging Computer Vision for Emergency Vehicle Detection-Implementation and Analysis. In Proceedings of the 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kharagpur, India, 1–3 July 2020; pp. 1–6. [Google Scholar] [CrossRef]
  8. Raj, V.S.; Sai, J.V.; Yogesh, N.L.; Preetha, S.K.; Lavanya, R. Smart Traffic Control for Emergency Vehicles Prioritization using Video and Audio Processing. In Proceedings of the 2022 6th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 25–27 May 2022; pp. 1588–1593. [Google Scholar] [CrossRef]
  9. Maligalig, K.C.; Amante, A.D.; Tejada, R.R.; Tamargo, R.S.; Santiago, A.F. Machine Vision System of Emergency Vehicle Detection System Using Deep Transfer Learning. In Proceedings of the 2022 International Conference on Decision Aid Sciences and Applications (DASA), Chiangrai, Thailand, 23–25 March 2022; pp. 1464–1468. [Google Scholar] [CrossRef]
  10. Ojha, A.; Sahu, S.P.; Dewangan, D.K. Vehicle Detection through Instance Segmentation using Mask R-CNN for Intelligent Vehicle System. In Proceedings of the 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 6–8 May 2021; pp. 954–959. [Google Scholar] [CrossRef]
  11. Eslami, M.; Faez, K. Automatic traffic monitoring using neural networks from satellite images. In Proceedings of the 2010 11th International Conference on Control Automation Robotics & Vision, Singapore, 7–10 December 2010; pp. 1616–1621. [Google Scholar] [CrossRef]
  12. Ghandour, A.J.; Krayem, H.A.; Jezzini, A.A. Autonomous Vehicle Detection and Classification in High Resolution Satellite Imagery. In Proceedings of the 2018 International Arab Conference on Information Technology (ACIT), Werdanye, Lebanon, 28–30 November 2018; pp. 1–5. [Google Scholar] [CrossRef]
  13. Alghazo, J.; Bashar, A.; Latif, G.; Zikria, M. Maritime Ship Detection using Convolutional Neural Networks from Satellite Images. In Proceedings of the 2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT), Bhopal, India, 18–19 June 2021; pp. 432–437. [Google Scholar] [CrossRef]
  14. Hinz, S. Detection and counting of cars in aerial images. In Proceedings of the 2003 International Conference on Image Processing (Cat. No.03CH37429), Barcelona, Spain, 14–17 September 2003; p. III-997. [Google Scholar] [CrossRef]
  15. Shi, F.; Zhang, T.; Zhang, T. Orientation-Aware Vehicle Detection in Aerial Images via an Anchor-Free Object Detection Approach. IEEE Trans. Geosci. Remote Sens. 2021, 59, 5221–5233. [Google Scholar] [CrossRef]
  16. Holla, A.; Verma, U.; Pai, R.M. Efficient Vehicle Counting by Eliminating Identical Vehicles in UAV aerial videos. In Proceedings of the 2020 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER), Udupi, India, 30–31 October 2020; pp. 246–251. [Google Scholar] [CrossRef]
  17. Bar, D.E.; Raboy, S. Moving Car Detection and Spectral Restoration in a Single Satellite WorldView-2 Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2013, 6, 2077–2087. [Google Scholar] [CrossRef]
  18. Zhu, J.; Sun, K.; Jia, S.; Li, Q.; Hou, X.; Lin, W.; Liu, B.; Qiu, G. Urban Traffic Density Estimation Based on Ultrahigh-Resolution UAV Video and Deep Neural Network. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 4968–4981. [Google Scholar] [CrossRef]
  19. Zhou, H.; Wei, L.; Lim, C.P.; Nahavandi, S. Robust Vehicle Detection in Aerial Images Using Bag-of-Words and Orientation Aware Scanning. IEEE Trans. Geosci. Remote Sens. 2018, 56, 7074–7085. [Google Scholar] [CrossRef]
  20. Yu, Y.; Gu, T.; Guan, H.; Li, D.; Jin, S. Vehicle Detection From High-Resolution Remote Sensing Imagery Using Convolutional Capsule Networks. IEEE Geosci. Remote Sens. Lett. 2019, 16, 1894–1898. [Google Scholar] [CrossRef]
  21. ElMikaty, M.; Stathaki, T. Car Detection in Aerial Images of Dense Urban Areas. IEEE Trans. Aerosp. Electron. Syst. 2018, 54, 51–63. [Google Scholar] [CrossRef]
  22. ElMikaty, M.; Stathaki, T. Detection of Cars in High-Resolution Aerial Images of Complex Urban Environments. IEEE Trans. Geosci. Remote Sens. 2017, 55, 5913–5924. [Google Scholar] [CrossRef]
  23. Chen, Z.; Wang, C.; Luo, H.; Wang, H.; Chen, Y.; Wen, C.; Yu, Y.; Cao, L.; Li, J. Vehicle Detection in High-Resolution Aerial Images Based on Fast Sparse Representation Classification and Multiorder Feature. IEEE Trans. Intell. Transp. Syst. 2016, 17, 2296–2309. [Google Scholar] [CrossRef]
  24. Chen, Z.; Wang, C.; Wen, C.; Teng, X.; Chen, Y.; Guan, H.; Luo, H.; Cao, L.; Li, J. Vehicle Detection in High-Resolution Aerial Images via Sparse Representation and Superpixels. IEEE Trans. Geosci. Remote Sens. 2016, 54, 103–116. [Google Scholar] [CrossRef]
  25. Moranduzzo, T.; Melgani, F. Car speed estimation method for UAV images. In Proceedings of the 2014 IEEE Geoscience and Remote Sensing Symposium, Quebec City, QC, Canada, 13–18 July 2014; pp. 4942–4945. [Google Scholar] [CrossRef]
  26. Chen, X.; Xiang, S.; Liu, C.-L.; Pan, C.-H. Vehicle Detection in Satellite Images by Parallel Deep Convolutional Neural Networks. In Proceedings of the 2013 2nd IAPR Asian Conference on Pattern Recognition, Naha, Japan, 5–8 November 2013; pp. 181–185. [Google Scholar] [CrossRef]
  27. Nguyen, T.T.; Grabner, H.; Bischof, H.; Gruber, B. On-line Boosting for Car Detection from Aerial Images. In Proceedings of the 2007 IEEE International Conference on Research, Innovation and Vision for the Future, Hanoi, Vietnam, 5–9 March 2007; pp. 87–95. [Google Scholar] [CrossRef]
  28. Lim, H.-S.; Park, H.-M.; Lee, J.-E.; Kim, Y.-H.; Lee, S. Lane-by-Lane Traffic Monitoring Using 24.1 GHz FMCW Radar System. IEEE Access 2021, 9, 14677–14687. [Google Scholar] [CrossRef]
  29. Zhang, Z.; Wang, X.; Huang, D.; Fang, X.; Zhou, M.; Zhang, Y. MRPT: Millimeter-Wave Radar-Based Pedestrian Trajectory Tracking for Autonomous Urban Driving. IEEE Trans. Instrum. Meas. 2022, 71, 8000117. [Google Scholar] [CrossRef]
  30. Fatseas, K.; Bekooij, M.J.G. Neural Network Based Multiple Object Tracking for Automotive FMCW Radar. In Proceedings of the 2019 International Radar Conference (RADAR), Toulon, France, 23–27 September 2019; pp. 1–5. [Google Scholar] [CrossRef]
  31. Ikram, M.Z.; Ali, M. 3-D object tracking in millimeter-wave radar for advanced driver assistance systems. In Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing, Austin, TX, USA, 3–5 December 2013; pp. 723–726. [Google Scholar] [CrossRef]
  32. Hussain, M.I.; Azam, S.; Munir, F.; Khan, Z.; Jeon, M. Multiple Objects Tracking using Radar for Autonomous Driving. In Proceedings of the 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), Vancouver, BC, Canada, 9–12 September 2020; pp. 1–4. [Google Scholar] [CrossRef]
  33. Manjunath, A.; Liu, Y.; Henriques, B.; Engstle, A. Radar Based Object Detection and Tracking for Autonomous Driving. In Proceedings of the 2018 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM), Munich, Germany, 15–17 April 2018; pp. 1–4. [Google Scholar] [CrossRef]
  34. Otto, C.; Gerber, W.; León, F.P.; Wirnitzer, J. A Joint Integrated Probabilistic Data Association Filter for pedestrian tracking across blind regions using monocular camera and radar. In Proceedings of the 2012 IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain, 3–7 June 2012; pp. 636–641. [Google Scholar] [CrossRef]
  35. Xu, G.; Shan, G.; Duan, X. Sensor scheduling for ground maneuvering target tracking in presence of detection blind zone. J. Syst. Eng. Electron. 2020, 31, 692–702. [Google Scholar] [CrossRef]
Figure 1. Schematic diagram of LLA coordinate system.
Figure 1. Schematic diagram of LLA coordinate system.
Sensors 22 09371 g001
Figure 2. Schematic diagram of ECEF coordinate system.
Figure 2. Schematic diagram of ECEF coordinate system.
Sensors 22 09371 g002
Figure 3. Schematic diagram of conversion from ENU coordinate system to radar coordinate system.
Figure 3. Schematic diagram of conversion from ENU coordinate system to radar coordinate system.
Sensors 22 09371 g003
Figure 4. Radar blind zone on a curved road.
Figure 4. Radar blind zone on a curved road.
Sensors 22 09371 g004
Figure 5. Combined with the high-precision map tracking algorithm.
Figure 5. Combined with the high-precision map tracking algorithm.
Sensors 22 09371 g005
Figure 6. Coverage of traffic radar.
Figure 6. Coverage of traffic radar.
Sensors 22 09371 g006
Figure 7. Radar installation in the actual scene.
Figure 7. Radar installation in the actual scene.
Sensors 22 09371 g007
Figure 8. Radar blind zone on a straight road.
Figure 8. Radar blind zone on a straight road.
Sensors 22 09371 g008
Figure 9. Radar track filtering results on the straight road. (a) Traditional method filtering results; (b) track filtering results combined with high-precision map.
Figure 9. Radar track filtering results on the straight road. (a) Traditional method filtering results; (b) track filtering results combined with high-precision map.
Sensors 22 09371 g009
Figure 10. Radar blind zone on a curved road.
Figure 10. Radar blind zone on a curved road.
Sensors 22 09371 g010
Figure 11. Radar track filtering results on the curved road. (a) Traditional method filtering results; (b) track filtering results combined with high-precision map.
Figure 11. Radar track filtering results on the curved road. (a) Traditional method filtering results; (b) track filtering results combined with high-precision map.
Sensors 22 09371 g011
Table 1. Traffic radar parameters.
Table 1. Traffic radar parameters.
ParametersShort-RangeMedium-RangeLong-Range
Installation inclination (elevation angle)
Installation inclination (azimuth angle)
Range45 m–120 m100 m–220 m200 m–550 m
Range resolution±0.2 m
FOV (elevation angle)±6°±4.4°±4.4°
FOV (azimuth angle)Covering 10 lanes (50 m)
Maximum velocity−250 km/h ~ +250 km/h
Velocity resolution±0.1 m/s
Installation height10 m
Central Frequency79 GHz
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

An, Q.; Cai, Y.; Zhu, J.; Wang, S.; Han, F. Multi-Target Tracking Algorithm Combined with High-Precision Map. Sensors 2022, 22, 9371. https://doi.org/10.3390/s22239371

AMA Style

An Q, Cai Y, Zhu J, Wang S, Han F. Multi-Target Tracking Algorithm Combined with High-Precision Map. Sensors. 2022; 22(23):9371. https://doi.org/10.3390/s22239371

Chicago/Turabian Style

An, Qingru, Yawen Cai, Juan Zhu, Sijia Wang, and Fengxia Han. 2022. "Multi-Target Tracking Algorithm Combined with High-Precision Map" Sensors 22, no. 23: 9371. https://doi.org/10.3390/s22239371

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop