Radar Based Detection and Classification of Vulnerable Road Users ^†

Abstract

Radar sensors accurately detect different objects; however, the reliable classification of these objects remains challenging. In this contribution, new approaches to extracting and interpreting unique spectral features of pedestrians and cyclists are proposed. Both methods use range-Doppler maps, which contain information on the distance to and the velocity of a detected object. The detections originate from the local dynamic of the moving (body) parts, and therefore can be used to reconstruct a unique movement pattern, which is represented by a time dependent velocity distribution. Machine learning algorithms can also be applied to the obtained time series in order to automate the classification task.

1. Introduction

A high level of mobility is one of the bases of a highly developed society and economy. Convenient traffic interchange makes it possible to get to any place quickly and easily. However, with accelerating development of mobility options, the corresponding infrastructure is becoming increasingly complex. In order to meet the mobility demand in the future, considerable optimization is required. One obstacle in the development of new concepts is the lack of movement profiles of different road users.

Collection of movement profiles using radar-based methods has drawn much attention. There are different models proposed to extract the useful information from radar backscattering; however, very often they are not straightforward and require expensive computational resources. The aim of this paper is to present a simple approach to extract the movement patterns of different road users (here pedestrians and cyclists) in order to enable classification based solely on radar measurements.

2. Experimental

In order to distinguish between various objects based on radar measurements, different approaches have been proposed [1,2,3]. The majority of studies deal with a Doppler-sensor, which operates at a constant frequency with an unmodulated transmit waveform. This sensor demodulates the Doppler-shift of the reflected electromagnetic wave due to a radial movement of a backscattering from a detected object. As these reflection centers move with different velocities (for example, the legs and torso of a pedestrian as shown in Figure 1a), a spectrogram containing the mentioned velocities can be calculated using a short time Fourier transform (ST-FT), which is quite an expensive operation with respect to computational resources. The output of a Doppler-sensor only contains information about the radial relative velocities of objects or reflection centers. Thus, no insights about the distance to the detected objects can be deduced. However, Doppler-sensors are very sensitive with respect to small movements of objects, resulting in micro-Doppler signatures [4]. This has led to the widespread use of these sensors when it comes to pedestrian detection [5,6,7].

In this work, for the detection and classification of the movement pattern of different road users, a radar sensor with a chirp sequence modulation was used. In contrast to a Doppler sensor, it measures the distance to static and moving objects using frequency modulated continuous wave (FMCW) modulation. The transmitting carrier frequency is modulated with a digitally generated linear ramp also known as a chirp. A chirp reflected from an object is received again after time t, which is proportional to the distance to the object. To measure the velocity, the radar sensor sends FMCW range chirps spaced at intervals of time T. These multiple chirps form a frame have the dimensions of MxN, where M is the number of chirps and N is the number of samples in a chirp. Two-dimensional fast Fourier transform (2D-FFT) is a basic signal processing method for the raw data (frames) of the sensor. The result of this signal processing is a so-called range-Doppler map, which contains the information of both distance to and relative velocity of a detected object. Furthermore, angular information together with the direction of arrival can be obtained if multiple receiving channels are evaluated.

For the data acquisition, a commercial radar sensor IWR1443BOOST in combination with the DCA1000EVM-board, both produced by Texas Instruments, was used [8]. The main modulation parameters that were set for the measurements can be found in

Table 1. Main parameters for the FMCW radar sensor used for the measurements.

Parameters	Values
Carrier frequency	77 GHz
Bandwidth	3 GHz
Frame repetition rate	100 ms
Chirp rate	255 #/s
Sample rate	255 #/s
Chirp duration	60 µs

.

Comparing a Doppler plot with a range-Doppler plot, one can clearly see the advantage of a Doppler sensor with respect to a movement pattern. It provides a clear gait pattern, which cannot be deduced from a range-Doppler plot as one can see in Figure 1. However, it has to be mentioned that a Doppler spectrum has advantages when detecting one person. If several objects/persons are detected (with a certain distance to each other), the interpretation of Doppler spectra might become ambiguous. In case of a range-Doppler spectrum, one can separate multiple objects in one frame and proceed with further signal processing. Therefore, for the discussed purpose of collecting different movement profiles on roads, a FMCW-sensor demonstrates unbeatable advantages when compared to a Doppler sensor.

3. Results and Discussion

A FMCW sensor has the advantage of providing various parameters that can comprehensively describe the movement of any object. In order to reconstruct a unique movement pattern, two approaches described below can be used depending on the requirements of further signal processing or classification tasks.

3.1. Proposed Approaches for Radar Measurement Interpretation

The approaches developed in this work are based on the evaluation of a series of consecutive radar frames of the FMCW sensor in order to mimic a Doppler plot, but still to have the access to all the parameters provided by a range-Doppler map. In the course of this work, two approaches have been developed. The first one is based on an image analysis. For this purpose, a sequence of radar frames is summed up using the so-called max hold approach. The idea behind this approach is to compare the values of the corresponding bins/pixels from two neighboring frames and preserve the bigger value, which will be compared in turn to the corresponding value of the next frame, whereas the smaller value will be discarded. This operation has to be repeated for all pixels in all frames of the sequence. Based on the experimental results, it has been found that a sequence of seven frames (which results from a measurement with a duration of 0.7 s using the radar settings provided in Table 1) is sufficient to distinguish between pedestrian and bicycle movements. The results of this operation can be seen in Figure 2. This approach can be used to represent the FMCW radar measurements similar to the one of a Doppler sensor in order to obtain a clear movement pattern. Further evaluation of the reconstructed pattern can be done using deep learning algorithms in order to distinguish between different road users. A simple neural network with four convolutional blocks successfully distinguishes four categories (bicycle, pedestrian, two pedestrians and empty measurement) after training on a small dataset of around 1000 images per category. The results are not shown here, but will be published elsewhere.

The second approach is based on a statistical analysis. Depending on the distance to the sensor, the radar backscattering will account only for a small part of the range–velocity bins. To reduce computational load, dimensionality reduction is required. This can be achieved by extracting a mean velocity weighted with the corresponding intensity. For this purpose, the range bin with the strongest reflection is found. Afterwards, the mean velocity is calculated according to Equation (1):

$${\mathrm{v}}_{\mathrm{mean}}=\frac{\sum \mathrm{v}\cdot \mathrm{I}\left(\mathrm{v}\right)}{\sum \mathrm{I}\left(\mathrm{v}\right)},$$

(1)

where v is the velocity value, I(v) is the corresponding intensity at the range bin with the max intensity. The operation has to be repeated for every frame of the measurement. It results in a time series of mean velocity distribution, which demonstrates unique features for different objects, as can be seen in Figure 3. A further evaluation of the time series in order to classify different moving objects can go in various directions, which have to be discussed separately.

3.2. Gait Pattern of a Pedestrian

The gait pattern of a pedestrian is determined by the reflections from the torso (dominant) and the contribution of the legs and the hands. Therefore, the pattern demonstrates clear periodic movements as can be seen in Figure 2a and Figure 3a.

3.3. Gait Pattern of a Cyclist

In contrast to a pedestrian, the gait of a cyclist is dominated by the torso reflections with minor contributions from the legs. This results in a movement pattern that resembles a straight line, as one can see in Figure 2b and Figure 3b.

3.4. Movement Analysis

Figure 4 demonstrates a velocity distribution, which describes the movement of a bicycle. The velocity values are positive, which indicates that the movement takes place towards the measuring sensor. In part 1, the bicycle moves with a constant velocity and starts decelerating in part 2. When an object is situated too close to the radar sensor, one can observe artifacts of the measurement resulting in an increased noise level. This situation can be seen in part 3. The bicycle continues moving, slightly accelerating in part 4.

4. Conclusions

In this work, new approaches to interpreting unique spectral signatures of pedestrians and cyclists are proposed. Range-Doppler maps, which result from the local dynamic of the moving parts, are used to extract statistical parameters of the movement patterns. The gait pattern is therefore represented by a time-dependent velocity distribution. An image-based analysis of the range-Doppler maps are meant to replicate a gait pattern similar to the one obtained from a Doppler-sensor.