Path Tracking for Car-like Robots Based on Neural Networks with NMPC as Learning Samples

Abstract

In the field of path tracking for car-like robots, although nonlinear model predictive control (NMPC) can handle the system constraints well, its real-time performance is poor. To solve this problem, a neural network control method with NMPC as the learning sample is proposed. The design process of this control method includes establishing the NMPC controller based on the time-varying local model, generating learning samples based on this NMPC controller, and training to obtain the neural network controller. The proposed controller is tested by a joint simulation of MATLAB and Carsim and compared with other controllers. According to the simulation results, the accuracy of the NN controller is close to that of the NMPC controller and far better than that of the Stanley controller. In all simulations, the absolute value of displacement error of the NN controller does not exceed 0.2854 m, and the absolute value of heading error does not exceed 0.2279 rad. In addition, the real-time performance of the NN controller is better than that of the NMPC controller. The maximum time cost and average time cost of the NN controller are, respectively, 40.91% and 22.37% smaller than those of the NMPC controller under the same conditions.

1. Introduction

Car-like robots are a common type of wheeled mobile robot. For mobile robots, path tracking is often an essential feature. The function of path tracking is to make the robot travel along a reference path and reduce the deviation between its travel path and the reference path. The path tracking of car-like robots has some similarities with the path tracking of driverless vehicles, such as the work of Ortega et al. [1], but it also faces different challenges. In general, car-like robots typically travel at low speeds, but track reference paths that have large curvature and dramatic curvature changes. Therefore, some researchers, such as Naderi Samani et al., are interested in the path tracking of car-like robots and have published some papers on this topic [2,3,4,5,6,7,8,9,10,11,12]. In 2016, Naderi Samani et al. designed controllers for the parallel parking path tracking of car-like robots based on feedback linearization control and sliding mode control, respectively [2]. Hwang proposed a path tracking model and algorithm for car-like robots based on hierarchical variable structure control [3]. Abd Latip and Omar developed proportional-based and proportional-differential-based path tracking controllers for car-like robots and evaluated the performance of both controllers [4]. In 2018, Ballinas et al. proposed an automatic parallel parking system for a car-like robot, using fuzzy control for the path tracking part [5]. Ghaffari and Homaeinezhad also proposed a path tracking method for car-like robots based on fuzzy control [6]. In 2019, Ljungqvist et al. developed a path tracking system for systems with a car-like tractor and two trailers to solve the reversing challenges of such multi-body systems [7]. Kamran et al. proposed a reinforcement learning-based path tracking algorithm for car-like robots [8]. Wu developed an algorithm for path tracking of car-like robots under saturated input and sudden pulse disturbance [9]. In 2020, Mohan Rayguru et al. designed a sliding mode controller based on a robust observer to achieve path tracking for car-like robots in the presence of missing state quantities and faulty sensors [10]. Prokopyev and Sofronova compare several path tracking methods for car-like robots [11]. In 2021, Elobaid et al. proposed a transverse feedback linearization-based path tracking method for car-like robots [12]. In 2022, Yeom proposed a vehicle-like robot path tracking method using deep neural networks to adjust parameters [13]. There is no doubt that these research efforts are positive. However, these studies of car-like robots have not considered the effect of system constraints.

For the path tracking of driverless vehicles, Gong et al. pointed out that the system constraints, such as the front wheel turning angle constraint and front wheel turning angle speed constraint of the vehicle, have an impact on the path tracking [14]. Recently, several review papers pointed out the advantages of model predictive control (MPC) in dealing with system constraints [15,16,17,18,19]. Further, Bai et al. also pointed out that among multiple types of MPC, nonlinear MPC (NMPC) can more effectively utilize the reference path information in front of the robot to obtain better path tracking accuracy relative to other types of MPC [15,17,18]. Benefiting from the above advantages, NMPC has achieved a vast range of applications in path tracking. In 2017, Wang et al. proposed an MPC path tracking method for field vehicles [20]. Between 2019 and 2022, based on NMPC, Bai et al. attempted to solve the problems faced in path tracking for a variety of mobile devices, such as tractor-trailer vehicles [21], car-like vehicles [22,23,24], and differential wheeled robots [25,26]. In 2020, Yin et al. designed an NMPC-based path tracking controller using discrete previewed points in the inertial system [27]. Barzegar et al. used NMPC to solve the trajectory tracking problem for driverless vehicles [28]. In 2021, Farag developed an easy-to-port NMPC-based path tracking controller for driverless vehicles [29]. Chen et al. proposed an NMPC-based controller to improve path tracking accuracy and vehicle driving stability [30]. Franco et al. proposed a driverless system that includes lane line recognition and path tracking, where the path tracking is based on NMPC [31]. In 2022, Hang et al. proposed a driverless vehicle control framework that includes longitudinal and lateral control using NMPC [32]. Lee and Choi designed an NMPC-based path tracking controller with integrated braking and steering [33]. These studies also show that NMPC has significant advantages in path tracking accuracy and dealing with system constraints.

However, due to high computation costs, NMPC has the significant drawback of poor real-time performance. Bai et al. investigated the relationship between the parameters of NMPC and real-time performance to solve the real-time problem and pointed out that the real-time performance of the NMPC-based controller is better when the control horizon is small [34,35]. Wang et al. also attempted to optimize the real-time performance of the NMPC-based path tracking controller using continuous/generalized minimum residual (C/GMRES) [36]. Unfortunately, even after optimization, the computation cost of NMPC-based controllers is still high, limiting the application of such controllers. The work of Gómez-Ortega et al. provides further inspiration. In 1994, they pioneered using an NMPC-based controller as a learning sample to train a neural network controller [37]. In 1996, based on their previous work, they proposed a neural network control algorithm for two-wheeled differential robots considering obstacle information [38,39]. In 2019, Liu further investigated the effect of input layer parameters and network type on the performance of the path tracking controller [40]. The performance of these controllers in terms of real-time performance is exhilarating, but it is slightly unfortunate that these controllers are still weaker than the NMPC controller in terms of accuracy. By analyzing the above research, we found that the learning samples they use usually contain information about only a single reference point on the reference path. However, NMPC can consider information about multiple previewed points on the reference path in front of the robot. The difference in the number of reference points may be the reason for the inferior accuracy performance of these neural network controllers compared to the NMPC-based controllers.

Therefore, we conducted the following study to obtain a path tracking control method that can combine both accuracy and real-time. First, we analyzed the relationship between the reference point in front of the robot and the coordinate system. Second, we built NMPC-based path tracking controllers as learning samples. Finally, we designed the input and output of the neural network, built the neural network controller, and tested it with a joint simulation of MATLAB and Carsim. In summary, our contribution in this paper is to propose a neural network path tracking control method capable of considering multiple reference points on the forward reference path of a car-like robot.

2. Reference Path and Coordinate System

For path tracking, the reference path is very important. Most current research on path tracking gives the reference path in the global coordinate system. However, as shown in the upper part of Figure 1, the heading and coordinates of the reference path may be different even if the relative positions of the car-like robot and the reference path are the same. These differences are not conducive to finding a correspondence between the reference path and the control inputs generated by the path tracking controller. Converting the reference path from the global coordinate system into the car-like robot coordinate system can solve the above problem. As shown in the lower part of Figure 1, the two reference paths on the left and right sides of Figure 1, which are entirely different in the global coordinate system, are the same when transferred into the robot coordinate system.

Therefore, the reference points on the reference path that need to be considered in each control period need to be transferred to the robot coordinate system. The starting point of these reference points is the closest point on the reference path to the car-like robot. The reference points are then taken further ahead of the car-like robot. The length of the reference path between each reference point and the next reference point is equal to the product of the set speed of the car-like robot and the control period.

Then, the reference points in the global coordinate system can be obtained:

$$\begin{array}{cc}{x}_{\mathrm{rG}}\left(i|t\right)={\left[\begin{array}{ccc}{X}_{\mathrm{rG}}\left(i|t\right)& Y_{\mathrm{rG}}\left(i|t\right)& {\theta }_{\mathrm{rG}}\left(i|t\right)\end{array}\right]}^{\mathrm{T}},& i=1,2,\cdots ,n\end{array}$$

(1)

where $x$ is the reference point vector, $X$ is the horizontal coordinate, $Y$ is the vertical coordinate, $\theta$ is the heading angle, the subscript $\mathrm{r}$ indicates the information on the reference path, and the subscript $\mathrm{G}$ indicates the information in the global coordinate system, $\left(i|t\right)$ denotes the $i$-th message at moment $t$, and $n$ denotes the prediction horizon of the NMPC controller.

In each control period, the positioning system provides the coordinates and heading angle of the car-like robot in the global coordinate system, so the position information of the car-like robot can be expressed as:

$$x_{\mathrm{G}}\left(t\right)={\left[\begin{array}{ccc}{X}_{\mathrm{G}}\left(t\right)& Y_{\mathrm{G}}\left(t\right)& {\theta }_{\mathrm{G}}\left(t\right)\end{array}\right]}^{\mathrm{T}}$$

(2)

Then, the coordinates of the reference points in (1) are transformed:

$$\begin{array}{ll}\{\begin{array}{l}{X}_{\mathrm{r}}\left(i|t\right)=\left(X_{\mathrm{rG}}\left(i|t\right)-X_{\mathrm{G}}\left(t\right)\right)\cos {\theta }_{\mathrm{G}}\left(t\right)+\left(Y_{\mathrm{rG}}\left(i|t\right)-Y_{\mathrm{G}}\left(t\right)\right)\sin {\theta }_{\mathrm{G}}\left(t\right)\\ Y_{\mathrm{r}}\left(i|t\right)=\left(Y_{\mathrm{rG}}\left(i|t\right)-Y_{\mathrm{G}}\left(t\right)\right)\cos {\theta }_{\mathrm{G}}\left(t\right)-\left(X_{\mathrm{rG}}\left(i|t\right)-X_{\mathrm{G}}\left(t\right)\right)\sin {\theta }_{\mathrm{G}}\left(t\right)\\ {\theta }_{\mathrm{r}}\left(i|t\right)={\theta }_{\mathrm{rG}}\left(i|t\right)-{\theta }_{\mathrm{G}}\left(t\right)\end{array},& i=1,2,\cdots ,n\end{array}$$

(3)

The reference points in the car-like robot coordinate system can be abstracted as:

$$\begin{array}{cc}{x}_{\mathrm{r}}\left(i|t\right)={\left[\begin{array}{ccc}{X}_{\mathrm{r}}\left(i|t\right)& Y_{\mathrm{r}}\left(i|t\right)& {\theta }_{\mathrm{r}}\left(i|t\right)\end{array}\right]}^{\mathrm{T}},& i=1,2,\cdots ,n\end{array}$$

(4)

3. NMPC Controller Based on the Time-Varying Local Model

The NMPC controller design process is well known, so this section will be as brief as possible. The basis of all NMPC controllers is a mathematical model, and the mathematical model used here is a two-degree-of-freedom dynamics model of the car-like robot, where the input to the model is the front wheel turning angle:

$$u=\left[\delta \right]$$

(5)

The output is:

$$x={\left[\begin{array}{ccccc}X& Y& \theta & v_y& \omega \end{array}\right]}^{\mathrm{T}}$$

(6)

where $v_y$ is the lateral speed and $\omega$ is the heading angle speed.

A more detailed modeling process can be found in our previous work [24], the tire model of the car-like robot model uses the Magic Formula [41,42], and the final abstract model can be expressed as follows:

$$\dot{x}=f\left(x,u\right)$$

(7)

The prediction model for the NMPC controller is obtained by discretizing (7):

$$\{\begin{array}{l}x\left(1|t\right)=x\left(0|t\right)+Tf\left(x\left(0|t\right),u\left(1|t\right)\right)\\ ⋮\\ x\left(i|t\right)=x\left(i-1|t\right)+Tf\left(x\left(i-1|t\right),u\left(i|t\right)\right)\\ ⋮\\ x\left(m|t\right)=x\left(m-1|t\right)+Tf\left(x\left(m-1|t\right),u\left(m|t\right)\right)\\ ⋮\\ x\left(n|t\right)=x\left(n-1|t\right)+Tf\left(x\left(n-1|t\right),u\left(m|t\right)\right)\end{array}$$

(8)

where $m$ is the control horizon, $T$ is the control period, and in particular, $x\left(0|t\right)$ is the starting pose of the prediction model.

The most significant difference between the NMPC controller based on the time-varying local model and the traditional NMPC controller is the difference in the starting poses of the prediction model. In the traditional NMPC controller, the starting pose is the current pose of the car-like robot in the global coordinate system. In contrast, in the NMPC controller based on the time-varying local model, the starting pose is:

$$x\left(0|t\right)={\left[\begin{array}{ccccc}0& 0& 0& v_y\left(t\right)& \omega \left(t\right)\end{array}\right]}^{\mathrm{T}}$$

(9)

where $v_y\left(t\right)$ is the current lateral speed of the car-like robot and $\omega \left(t\right)$ is the current heading angle speed of the car-like robot.

Bringing (9) into (8), we can obtain all the predicted positional states of the car-like robot in the prediction horizon. Since the reference path has only three-dimensional information, the output vector of (9) can be set as:

$$\begin{array}{cc}{x}_{\mathrm{o}}\left(i|t\right)={\left[\begin{array}{ccc}{X}_{\mathrm{o}}\left(i|t\right)& Y_{\mathrm{o}}\left(i|t\right)& {\theta }_{\mathrm{o}}\left(i|t\right)\end{array}\right]}^{\mathrm{T}},& i=1,2,\cdots ,n\end{array}$$

(10)

where the subscript $\mathrm{o}$ indicates the output information.

The relationship between the output state and the predicted state is:

$$x_{\mathrm{o}}\left(t+i|t\right)\subset x\left(t+i|t\right),i=1,2,\cdots ,n$$

(11)

The optimization objective function of the NMPC controller can be designed as:

$$\begin{array}{ll}\hfill & \underset{u\left(i|t\right)}{\min }J={{\displaystyle \sum _{i=1}^n\Vert x_{\mathrm{o}}\left(i|t\right)-x_{\mathrm{r}}\left(i|t\right)\Vert }}_Q^2+{{\displaystyle \sum _{i=0}^{m-1}\Vert u\left(i+1|t\right)-u\left(i|t\right)\Vert }}_R^2\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & -{\delta }_{\lim }\le \delta \left(i\right)\le {\delta }_{\lim }\hfill \\ \hfill & -T{\dot{\delta }}_{\lim }\le \Delta \delta \left(i\right)\le T{\dot{\delta }}_{\lim }\hfill \end{array}$$

(12)

where $Q$ and $R$ are weight matrices and the subscript $\lim$ denotes the limit.

The first value of the input sequence obtained by solving (12) is the control input in this control period.

4. Neural Network Controller with NMPC as Learning Samples

Due to the success of feed-forward neural networks in numerous fields [43,44], we chose this neural network as the basis for building the controller. The training principle of the neural network path tracking controller is shown in Figure 2.

The NMPC controller in the joint simulation of MATLAB and Carsim generates the learning samples. The input and output of the neural network are the same as those of the NMPC controller. The information entered into the NMPC controller includes the front wheel turning angle before the start of the control period and the positional state of reference points on the reference path in the robot coordinate system:

$$\xi ={\left[\begin{array}{ccccccc}\delta \left(0|t\right)& x_{\mathrm{r}}{\left(1|t\right)}^{\mathrm{T}}& x_{\mathrm{r}}{\left(2|t\right)}^{\mathrm{T}}& \cdots & x_{\mathrm{r}}{\left(i|t\right)}^{\mathrm{T}}& \cdots & x_{\mathrm{r}}{\left(n|t\right)}^{\mathrm{T}}\end{array}\right]}^{\mathrm{T}}$$

(13)

The output information is the front wheel turning angle generated during that control period:

$$\zeta =\left[\delta \left(1|t\right)\right]$$

(14)

Considering that a neural network with two hidden layers can theoretically fit arbitrary functions [45], we can set the number of hidden layers of the neural network to 2. The structure of the neural network controller is shown in Figure 3.

The number of nodes in the hidden layer can be calculated by the empirical formula:

$$\{\begin{array}{l}j=\sqrt{\alpha +\beta }+{\lambda }_j\\ k=\sqrt{\alpha +\beta }+{\lambda }_k\end{array}$$

(15)

where $\alpha$ is the number of input layer nodes with a value of $3n+1$, $\beta$ is the number of output layer nodes with a value of 1, and ${\lambda }_j$ and ${\lambda }_k$ are the practical adjuster.

In addition, since the neural network controller cannot handle constraints strictly, the range of constraints of the simulated system can be reduced when generating samples to reduce the impact of system constraints on the neural network controller:

$$\{\begin{array}{l}{\delta }_{\mathrm{slim}}={\eta }_1{\delta }_{\lim }\\ {\dot{\delta }}_{\mathrm{slim}}={\eta }_2{\dot{\delta }}_{\lim }\end{array}$$

(16)

where ${\eta }_1$ and ${\eta }_1$ are range reduction factors.

5. Joint Simulation and Results

The joint simulation software includes MATLAB R2020a and Carsim 2017.1. Carsim provides the controlled car-like robot model with the parameters shown in

Table 1. Parameters of the car-like robot.

Parameter	Value	Parameter	Value
Wheelbase	1 m	Distance from the center of mass to the front axle	0.5 m
Mass	1230 kg	Rotational inertia around the vertical direction	1343.1 kg∙m−2
${\delta }_{\lim }$	30°	Ground adhesion coefficient	0.8
${\dot{\delta }}_{\lim }$	15°/s	Tire type	185/65 R15

. The simulation operates on a Dell G15 5510 laptop computer with an Intel(R) Core(TM) i5-10200H 2.40 GHz processor, 16 GB of RAM, and 512 GB of a solid-state drive. The control period is set to 20 ms.

The simulations are divided into three groups. The first group is used to compare the performance of the proposed controller with other controllers under the same condition, and the second and third groups are used to test the performance of the proposed controller under different conditions.

5.1. Comparison with Other Controllers

The proposed neural network (NN) controller is compared with the NMPC and Stanley controller in this set of simulations. The parameters of the NN controller and NMPC controller are shown in

Table 2. Parameters of the controllers.

Parameter	Value	Parameter	Value
$n$	30	$m$	2
Q	$diag\left\{10,10,1\right\}$	$R$	[0.01]
$j$	20	$k$	10
${\eta }_1$	0.85	${\eta }_2$	0.73

. The data generated by NMPC tracking reference paths are the learning samples of NN. The interval between every two data sets is 0.05 s, and there are 884 sets. The gain of the Stanley controller is 2. The speed of the car-like robot is 3 m/s.

Figure 4a shows the reference path, which is a U-shaped curve. U-turns are a common car-like robot work path and are a key component of complex paths. We generate the reference path in the form of a discrete point column, specifically, the coordinates and heading of each reference point on the reference path employing a function whose independent variable is the mileage. Figure 4b shows the driving trajectory of the car-like robot. It can be seen from the figure that both the NN and NMPC controller can control the car-like robot to track the reference path, but the Stanley controller fails.

Figure 5 shows the errors. The NN controller and the NMPC controller have minor errors and eventually converge. The maximum absolute values of displacement error and heading error of the NN controller are 0.1314 m and 0.2186 rad, respectively. These values for the NMPC controller are 0.1181 m and 0.2137 rad. It can be seen that the NN controller has very similar accuracy to the NMPC controller. However, in the same conditions, the displacement error of the Stanley controller is significantly divergent, and the Stanley controller is considered to fail when its displacement error is too large.

Figure 6 and Figure 7 show the front wheel turning angle (FWTA) and the front wheel turning angle speed (FWTAS), respectively. The FWTA generated by the NN controller is close to that executed by the system. The FWTA generated by the NMPC controller is the same as that executed by the system, and the FWTA generated by the Stanley controller and executed by the system have significant differences. It is further known from the FWTAS that the control inputs generated by the NN controller will slightly exceed the front wheel turning angle speed constraint. The control inputs generated by the NMPC controller will not exceed it, and those generated by the Stanley controller will exceed it significantly. Based on these phenomena, it is clear that the NMPC controller can handle the system constraints well, the NN controller is less affected by the system constraints, and the Stanley controller is highly affected by the system constraints.

Figure 8 shows the real-time performance, i.e., the computation time of each controller in each control period and its statistical values. Intuitively, the real-time performance of the NN controller is better than that of the NMPC controller. Specifically, in each control period, the maximum time cost of the NN controller is 13 ms, and the average time cost is 8.19 ms, which are 40.91% and 22.37% less than those values of the NMPC controller, respectively. In addition, by combining Figure 8a and Figure 6, we can see that the NMPC controller has a considerable time cost during the turn. Even in some control cycles, the time cost exceeds the control period. Although this situation has little impact on the simulation, in the actual control system, it means that the car-like robot cannot receive the control input generated by the controller in that control period. Therefore, there is a significant risk of loss of control. Thus, in summary, the NN controller significantly outperforms the NMPC controller in real-time performance.

Of course, the Stanley controller has better real-time performance than the NN controller. Still, the massive disadvantage in the accuracy of the Stanley controller prevents it from winning the competition with the NN controller.

5.2. Simulation at Different Speeds

This set of simulations is designed to test the performance of the NN controller at different speeds. The speed of the car-like robot is set to 2 m/s, 3 m/s, and 4 m/s, respectively. Considering the different optimal prediction horizons at different speeds [22], the prediction horizon is set to 28 for a speed of 2 m/s and 50 for a speed of 4 m/s.

The trajectory of the simulation at different speeds is shown in Figure 9. According to Figure 9, the speed variation has a negligible effect on the NN controller.

The error is shown in Figure 10. The displacement error and heading error of the NN controller are small and eventually converge at different speeds. The absolute value of the displacement error does not exceed 0.2854 m, and the absolute value of the heading error does not exceed 0.2279 rad.

Figure 11 and Figure 12 show the FWTA and the FWTAS, respectively. As can be seen in the figure, the NN controller is less affected by the system constraints. In addition, the FWTAS generated from the NN controller is closer to that executed by the system when the speed is 4 m/s. So, it can be inferred that increasing the prediction horizon can further mitigate the effect of the system constraint on the NN controller.

Figure 13 shows the real-time performance. As can be seen from the figure, the real-time performance of the NN controller does not vary with speed and is less influenced by the prediction horizon. For three sets of simulations with different speeds, the maximum time cost in each control period is 14 ms, and the maximum value of the average computation cost is 8.487 ms.

5.3. Simulation When Tracking the Complex Reference Path

This set of simulations is designed to test the performance of the NN controller when tracking the complex reference path. The learning samples of NN in this set of simulations are the sum of the samples of a right turn and the samples of the first set of simulations. The reference path of the right turn samples is the same as that in the first set of simulations, except that the endpoint is used as the start point. The speed of the car-like robot is set to 2 m/s.

The trajectory of the simulation is shown in Figure 14, and it can be seen that the complex reference path has a negligible effect on the NN controller.

The error is shown in Figure 15. The displacement error and heading error of the NN controller are small and eventually converge at different speeds. The absolute value of the displacement error does not exceed 0.1330 m, and the absolute value of the heading error does not exceed 0.2186 rad.

Figure 16 shows the FWTA and the FWTAS, respectively. As can be seen in the figure, the NN controller is less affected by the system constraints.

As shown in Figure 17, the real-time performance of the NN controller is less influenced by the reference path. In this set of simulations, the maximum time cost in each control period is 12 ms and the maximum value of the average time cost is 8.181 ms.

6. Conclusions

To obtain a path tracking controller for car-like robots that can combine both accuracy and real-time, we proposed a neural network control method using an NMPC controller based on the time-varying local model as the learning sample. The controller was tested by joint simulation of MATLAB and Carsim and compared with other controllers. Based on the simulation results, the following conclusions can be obtained.

First, the proposed NN controller has high accuracy. The maximum absolute value of displacement error is 0.2854 m, and the maximum absolute value of heading error is 0.2279 rad in all simulations. The accuracy of the NN controller is very close to that of the NMPC controller under the same condition. Furthermore, the Stanley controller fails in this condition.

Second, the proposed NN controller has excellent real-time performance. In all simulations, the maximum time cost of this controller in each control period is 14 ms, and the average time cost does not exceed 8.487 ms. In the same condition, the maximum time cost and average time cost of the NN controller are 40.91% and 22.37% smaller than those of the NMPC, respectively.

In the near future, we will conduct further research on real-world robot experiments, including improving the robustness of the control method to perturbations and porting the algorithm to a car-like robot platform.