Estimation of Vehicle Attitude, Acceleration, and Angular Velocity Using Convolutional Neural Network and Dual Extended Kalman Filter

Abstract

The acceleration of a vehicle is important information in vehicle states. The vehicle acceleration is measured by an inertial measurement unit (IMU). However, gravity affects the IMU when there is a transition in vehicle attitude; thus, the IMU produces an incorrect signal output. Therefore, vehicle attitude information is essential for obtaining correct acceleration information. This paper proposes a convolutional neural network (CNN) for attitude estimation. Using sequential data of a vehicle’s chassis sensor signal, the roll and pitch angles of a vehicle can be estimated without using a high-cost sensor such as a global positioning system or a six-dimensional IMU. This paper also proposes a dual-extended Kalman filter (DEKF), which can accurately estimate acceleration/angular velocity based on the estimated roll/pitch information. The proposed method is validated by real-car experiment data and CarSim, a vehicle simulator. It accurately estimates the attitude estimation with limited sensors, and the exact acceleration/angular velocity is estimated considering the roll and pitch angle with de-noising effect. In addition, the DEKF can improve the modeling accuracy and can estimate the roll and pitch rates.

1. Introduction

In recent decades, the vehicle controller has been developed significantly for stability and user convenience. Typically, the electronic stability controller (ESC) or active roll stabilization (ARS) are used for ensuring vehicle stability in chassis controller, adaptive cruise control (ACC), or lane keeping system (LKS) as well as for ensuring convenience in advanced driver assistant system (ADAS). To improve the performance of these controllers, vehicle states must be estimated with a high accuracy. The commonly required and important state information of aforementioned controllers are acceleration and angular velocity. In a vehicle, the inertial measure unit (IMU) measures the acceleration and angular velocity using an inertial force. However, if a transition in attitude (roll and pitch) occurs, the gravitational force is reflected in the sensor value. The IMU cannot distinguish between gravitational force and inertial forces; therefore, a change in the attitude of the vehicle causes a fatal error in the IMU.

Several studies have been conducted to overcome the errors in these accelerometers and to estimate the exact state of the vehicle. In [1,2], the adaptive Kalman filter was designed to minimize the effect of the accelerometer offset errors. In addition, the Kalman filter was used to estimate the accelerometer offset and vehicle velocity [3,4,5]. However, the accuracy of estimating vehicle states was limited because of a lack of information of the vehicle attitude.

Several studies have been conducted to estimate the accurate vehicle attitude in various ways. Using IMU or GPS or vehicle dynamics, researchers tried to estimate the vehicle roll and pitch angle. In addition, due to the recent developments in artificial intelligence and neural networks, data-driven estimators have been used for state estimation [6]. These estimators have the advantage of having high accuracy by training the data directly in situations where accurate mathematical modeling is difficult. The literature review is discussed separately in Section 2.

This paper proposes a novel vehicle attitude estimator using a convolutional neural network (CNN), based on the advantages of a data-driven estimator. Figure 1 shows the architecture of the proposed algorithm. First, we select the features based on vehicle roll and pitch dynamics, importance judgment and sensor usability. By using the selected features sequentially [7], the three-dimensional IMU and vehicle sensor build a CNN-based regression model that can estimate the vehicle roll and pitch angles without GPS. Moreover, a dual extended Kalman filter (DEKF) is designed to estimate the exact acceleration/angular velocity based on the estimated roll and pitch angles with removing the effect of gravitational force. The proposed model is based on the sprung mass six-degree-of-freedom (6-DOF) model and increases the modeling accuracy while estimating the tire cornering stiffness simultaneously. The performance of the proposed algorithm was verified using real car experiment data and MATLAB/Simulink with CarSim, a vehicle simulator. A CNN conducted training and verification using data collected from various driving conditions as an experimental vehicle with RT-3002 which is a high-accuracy GPS-inertial navigation system (INS) from Oxford Technical Solutions Ltd. The performance of the DEKF was verified with a scenario in which postural changes occur using CarSim and MATLAB/Simulink.

The remainder of this paper is organized as follows. Section 2 describes the related literature review. Section 3 explains the methodology including neural network and DEKF. Section 4 describes the results of performance verification. Section 5 summarizes the study and contributions.

2. Literature Review

Several studies have been conducted to estimate the accurate vehicle attitude in various ways. In [8], authors proposed an observer which can estimate the land vehicle’s roll and pitch by using an IMU and the kinematics model. Also, the adaptive Kalman filter was proposed based on IMU aided by vehicle dynamics [9]. However, these methods have a low accuracy in dynamic situations. This problem was attempted to be solved in [10,11,12,13], based on the Kalman filter by compensating for the external acceleration that interfered with the estimation of the attitude, but the accuracy was limit.

Therefore, many researchers have proposed sensor fusion method using not only IMU but fusion with global positioning system (GPS). Representatively, the method in which IMU and GPS fusion with Kalman filter [14,15,16] or sliding mode observer [17] can be used. These methods can increase the accuracy, but they require high-cost sensors such as GPS and are highly affected by GPS performance.

Accordingly, vehicle attitude estimator studies were conducted without the use of GPS, based on the characteristics of the vehicle dynamics. Using vehicle roll dynamics, a dynamic observer design based on a reliable rollover index [18] was proposed. Also, the Kalman filter [19] and the robust observer [20] based on vehicle roll dynamics which estimate roll angle were proposed. Alternatively, using lateral dynamics, the research which estimate road bank angle [21,22] was proposed. However, these methods exhibited limited accuracy in transient situations. A method for estimating both roll and pitch attitudes using a six-dimensional IMU and bicycle model [23,24] was also studied. These methods exhibited a high accuracy, but they have a short validation range and required hard-to-get data such as six-dimensional IMU.

Several recent studies have also been conducted to estimate the vehicle roll and pitch angle using neural network. In [25,26,27], the vehicle roll angle estimation method using sensor fusion with a neural network and Kalman filter was proposed. Furthermore, a vehicle roll and road bank angle estimation method based on a deep neural network was introduced [28]. However, the pitch attitudes were not estimated, and the six-dimensional IMU data such as roll rate or vertical acceleration were required, which complicated the estimation. Literature reviews are summarized in

Table 1. Literature review summarizing.

References	Methodology	Model
[8]	Linear observer	IMU kinematic model
[9]	Kalman filter	IMU + Vehicle dynamics (bicycle model and wheel model)
[10,11,12,13]	Kalman filter	IMU external acceleration model
[14,15,16]	Kalman filter	IMU + GPS model
[17]	Sliding mode observer	IMU + GPS model
[18]	Dynamic observer	Vehicle roll dynamics
[19]	Kalman filter	Vehicle roll dynamics
[20]	Dual Kalman filter	Vehicle roll dynamics
[21,22]	Linear observer	Vehicle lateral dynamics (bicycle model)
[23,24]	Linear observer, Sliding mode observer	Vehicle lateral dynamics (bicycle model) + IMU
[25,26,27]	Kalman filter + neural network	Vehicle roll dynamics + fully connected layer
[28]	Neural network	Fully connected layer

.

3. Methodology

3.1. Data-Driven-Based Estimator (Neural Network) Design

3.1.1. Feature Selection

Feature selection is based on vehicle dynamics, sensor usability, and attention mechanism. The purpose of neural network is to design a regression model for calculating the roll and pitch angles, so select the primary feature candidates for the vehicle’s roll and pitch dynamics. Vehicle roll dynamics [29] can be written as:

$$\mathsf{\Sigma }{M}_x=h_R{m}_{s}gsin\varphi -\frac{1}{2}{k}_s{l}_s^{2}sin\varphi -\frac{1}{2}{b}_s{l}_s^2\dot{\varphi }sin\varphi +\mathsf{\Sigma }{F}_y\left(h_{CG}-h_R\right)$$

(1)

$$\mathsf{\Sigma }{F}_y=2C_{\alpha f}\left(\delta -\frac{v_y+l_f\dot{\psi }}{v_x}\right)+2C_{\alpha r}\left(-\frac{v_y-l_r\dot{\psi }}{v_x}\right)+m_{s}gsin\varphi .$$

(2)

where $M_x$ is the moment respect to the roll axis, $h_R$ is the height of the from center of gravity to roll center, $m_s$ is the vehicle sprung mass, $g$ is the gravity acceleration, $\varphi$ is the roll of vehicle, $k_s$ is the stiffness of the suspension, $l_s$ is the length of the wheel track, $b_s$ is the damping coefficient of the suspension, and $h_{CG}$ is the height of the center of gravity from the ground.

The lateral force applied to the vehicle can be expressed by (2) based on the bicycle model. $C_{\alpha f}$ and $C_{\alpha r}$ are the cornering stiffness values of each front and rear tire, respectively, $\delta$ is the steering angle, $v_x$ and $v_y$ are the velocities of each x and y axes, respectively, $l_f$ and $l_r$ are the length of each front and rear wheel axis from the center of gravity, and $\dot{\psi }$ is the yaw rate.

The vehicle pitch dynamics can be written as [29]:

$$\mathsf{\Sigma }{M}_y=-k_s{l}_f^{2}sin\theta -k_s{l}_r^{2}sin\theta -b_s{l}_f^2\dot{\theta }cos\theta -b_s{l}_r^2\dot{\theta }cos\theta -\left(\mathsf{\Sigma }{F}_{drive}-\mathsf{\Sigma }{F}_{brake}\right)h_{CG}$$

(3)

$$\mathsf{\Sigma }{F}_{drive}=\left[\frac{N_{tf}{\eta }_{tf}}{r_{eff}}{T}_e-\left\{\left(I_e+I_t\right)N_{tf}^2+I_d{N}_f^2+I_w\right\}\frac{a_x}{r_{eff}^2}\right]$$

(4)

$$\mathsf{\Sigma }{F}_{brake}=\frac{2P_{brake}{A}_{caliper}{\mu }_{pad}{R}_b}{r_{eff}}$$

(5)

where $M_y$ is the moment with respect to the pitch axis and $\theta$ is the pitch of the vehicle.

The traction force applied to the vehicle can be expressed by (4) based on the vehicle drive-line dynamics. $N_{tf}$ is the gear ratio of the transmission and differential gears, ${\eta }_{tf}$ is the efficiency of the power transfer from the engine to the wheel axis, $r_{eff}$ is the effective radius of the tire, $T_e$ is the engine torque, $I_e, I_t, I_d, I_w$ are the inertia of each engine, transmission, differential gear, and wheel, respectively, and $a_x$ is the longitudinal acceleration of the vehicle.

Brake force applied to the vehicle is given by (5). $P_{brake}$ is the pressure of the master cylinder, $A_{caliper}$ is the area of the brake pad caliper, ${\mu }_{pad}$ is the friction coefficient of the brake pad, and $R_b$ is the distance from the wheel center to the brake pad.

Based on (1)–(5), we can choose only variables, except for static parameters such as the vehicle mass. As a result, a total of 13 feature candidates were selected, including the acceleration and yaw rate. Subsequently, seven features were selected considering the availability of sensors in the experimental vehicle.

Table 2. Selected features considering sensor usability.

Feature Name	Description
Ax	Vehicle longitudinal acceleration from IMU
Ay	Vehicle lateral acceleration from IMU
Yaw Rate	Vehicle yaw rate from IMU
Brake Pres	Brake pressure from the master cylinder
Str Angle	Steering wheel angle
Throttle	Engine throttle valve opening degree (0~1)
$\mathsf{\Sigma }$WheelSpd (Vx)	Sum of the wheel rotation speed

shows the selected features. Based on these, we conducted an analysis of importance based on the attention mechanism [30]. The results of the importance analysis are presented in Appendix A. According to Appendix A, the final features were selected same as Table 2.

3.1.2. Network Design

Before configuring a 2-d input to the CNN, we calculate ${\varphi }_{static,pseudo}$ and ${\theta }_{static,pseudo}$ reflecting the static roll and pitch angles using feature sensor values. The pseudo values are written as:

$${\varphi }_{static,pseudo}=e^{-a·V_x}·{\sin }^{-1}\left(\frac{a_y}{g}\right)$$

(6)

$${\theta }_{static,pseudo}=e^{-b·V_x}{e}^{-c·{\theta }_{throttle}}{e}^{-\frac{d}{P_{brake}}}·{\sin }^{-1}\left(-\frac{a_x}{g}\right)$$

(7)

where a,b,c,d are the constant design parameters. (5) and (6) are similar to the road bank and the slope angle, respectively when the vehicle’s wheel speed is zero. If the vehicle’s wheel speed increases, (5) and (6) become zero.

The neural network architecture is shown in Figure 2. The network is composed of CNN part and fully connected layer (FCL) part in parallel configuration. The input array is composed of a mux of each sequential feature sensor data including (5), (6). The sequential information is 2 s for 0.01 s, therefore the input array size is (200$\times$9). CNN part uses all inputs, namely all-time series data in the past 2 s, but FCL part uses only the last row of the input array, meaning only current step data. This means that the CNN part is designed with the intention of estimating dynamic vehicle body changes while driving and FCL part is designed to estimate the roll and pitch angles in static scenarios. The CNN part comprises four convolution layers and two fully connected layers. The first layer converts the input matrix into a square matrix and shuffles the sensor placement order. Then, it passes through the three convolution layers, and then pass the one convolution layer with large size filter and wide strides for compressing the data. Next, unfold to fully connected layer and make the last layer’s size (256$\times$1). In FCL part, there are four layers and final layer’s size is also (256$\times$1). The two final layers are concatenated, and then the regression model is constructed, which calculates two outputs using one fully connected layer. Hyperparameters of the neural network such as the number of filters or activation function are described in Appendix B.

3.2. Dual Extended Falman Filter Design

3.2.1. State Space Model

The state space model is based on the vehicle 6-DOF sprung mass model [31,32] for expressing the six-dimensional acceleration and angular velocity. The dynamics are composed based on the Euler rigid body equation and the force or moment of each axis which is given by each dynamic [29]. Figure 3 shows the six-dimensional motion of vehicle sprung mass. The forces or moment of each axis are given by:

$$\mathsf{\Sigma }{F}_x=m_s\left(\dot{v_x}+v_z\dot{\theta }-v_y\dot{\psi }-h_R\ddot{\theta }+h_R\dot{\varphi }\dot{\psi }\right)$$

(8)

$$\mathsf{\Sigma }{F}_y=m_s\left(\dot{v_y}+v_x\dot{\psi }-v_z\dot{\varphi }-h_R\ddot{\varphi }+h_R\dot{\theta }\dot{\psi }\right)$$

(9)

$$\mathsf{\Sigma }{F}_z=m_s\left(\dot{v_z}+v_y\dot{\varphi }-v_x\dot{\theta }-h_R{\dot{\varphi }}^2-h_R{\dot{\theta }}^2\right)$$

(10)

$$\mathsf{\Sigma }{M}_x=I_x\ddot{\varphi }+\left(I_z-I_y\right)\dot{\theta }\dot{\psi }-m_s{h}_R\left(\dot{v_y}+v_x\dot{\psi }-v_z\dot{\varphi }-h_R\ddot{\varphi }+h_R\dot{\theta }\dot{\psi }\right)$$

(11)

$$\mathsf{\Sigma }{M}_y=I_y\ddot{\theta }+\left(I_x-I_z\right)\dot{\varphi }\dot{\psi }$$

(12)

$$\mathsf{\Sigma }{M}_z=I_z\ddot{\psi }+\left(I_y-I_x\right)\dot{\varphi }\dot{\theta }$$

(13)

where $\mathsf{\Sigma }{F}_x, \mathsf{\Sigma }{F}_y, \mathsf{\Sigma }{F}_z$ are the sum of forces of the x, y, and z axes, respectively. $\mathsf{\Sigma }{M}_x, \mathsf{\Sigma }{M}_y, \mathsf{\Sigma }{M}_z$ are the sum of moments of the x, y, and z axes, respectively. $I_x, I_y, I_z$ are the moments of inertia along the x, y, and z axes, respectively. $\mathsf{\Sigma }{F}_x, \mathsf{\Sigma }{F}_y, \mathsf{\Sigma }{F}_z, \mathsf{\Sigma }{M}_x, \mathsf{\Sigma }{M}_y,$ and $\mathsf{\Sigma }{M}_z$ can be derived from a vehicle model such as a bicycle model. Therefore, the nonlinear state space equation can be written as:

$${\dot{x}}_1=\frac{\left(\frac{N_{tf}{\eta }_{tf}}{r_{eff}}{u}_1-\mathsf{\Sigma }{F}_{brake}-\frac{1}{2}\rho C_{d}A{x}_1^2-m_{s}gsin\left(x_8\right)\right)}{\left\{m_s+\frac{\left(I_e+I_t\right)N_{tf}^2+I_d{N}_f^2+I_w}{r_{eff}^2}\right\}}-x_3{x}_5+x_2{x}_6+h_R\dot{x_5}-h_R{x}_4{x}_6$$

(14)

$${\dot{x}}_2=\frac{1}{m_s}\left\{\mathsf{\Sigma }{F}_y+m_{s}gsin\left(x_7\right)\right\}-x_1{x}_6+x_3{x}_4+h_R\dot{x_4}-h_R{x}_5{x}_6$$

(15)

$${\dot{x}}_3=\frac{1}{m_s}{k}_s\sin \left(x_8\right)\left(l_f-l_r\right)+b_s\cos \left(x_8\right)x_5\left(l_f-l_r\right)-x_2{x}_4+x_1{x}_5+h_R{x}_4^2+h_R{x}_5^2$$

(16)

$${\dot{x}}_4=\frac{1}{I_x+m_s{h}_R^2}\left\{h_R{m}_{s}gsin\left(x_7\right)-\frac{1}{2}{k}_s{l}_s^2\sin \left(x_7\right)-\frac{1}{2}{b}_s{l}_s^2{x}_4\cos \left(x_7\right)+\mathsf{\Sigma }{F}_y\left(h_{CG}-h_R\right)-\left(I_z-I_y\right)x_5{x}_6+m_s{h}_R\left({\dot{x}}_2+x_1{x}_6-x_3{x}_4+h_R{x}_5{x}_6\right)\right\}$$

(17)

$${\dot{x}}_5=\frac{1}{I_y}\left\{-\left(l_f^2+l_r^2\right)k_s\sin \left(x_8\right)-\left(l_f^2+l_r^2\right)b_s{x}_5\cos \left(x_8\right)-\left(\mathsf{\Sigma }{F}_{drive}-\mathsf{\Sigma }{F}_{brake}\right)h_{CG}-\left(I_x-I_z\right)x_5{x}_6\right\}$$

(18)

$${\dot{x}}_6=\frac{1}{I_z}\left\{-\frac{2l_f{C}_{\alpha f}-2l_r{C}_{\alpha r}}{x_1}{x}_2-\frac{2l_f^2{C}_{\alpha f}+2l_r^2{C}_{\alpha r}}{x_1}{x}_6+2l_f{C}_{\alpha f}{u}_3-\left(I_y-I_x\right)x_4{x}_5\right\}$$

(19)

$${\dot{x}}_7=x_4$$

(20)

$${\dot{x}}_8=x_5$$

(21)

$\mathsf{\Sigma }{F}_{drive}$, $\mathsf{\Sigma }{F}_{brake}$, and $\mathsf{\Sigma }{F}_y$ have been described previously. The state vector $x=\left[v_x v_y v_z \dot{\varphi } \dot{\theta } \dot{\psi } \varphi \theta \right]$, state input $u= \left[T_e P_{brake} \delta \right]$. $\rho$ is the density of the air, $C_d$ is the drag coefficient and $A$ is the frontal area of the vehicle.

State outputs include longitudinal acceleration, lateral acceleration, and yaw rate from IMU. The roll and pitch angles from the neural network are also included. Thus, the state output $y=\left[a_x a_y \dot{\psi } \varphi \theta \right]$ and it can be written as:

$$a_x=\frac{\left(\frac{N_{tf}{\eta }_{tf}}{r_{eff}}{u}_1-\mathsf{\Sigma }{F}_{brake}-\frac{1}{2}\rho C_{d}A{x}_1^2-m_{s}gsin\left(x_8\right)\right)}{\left\{m_s+\frac{\left(I_e+I_t\right)N_{tf}^2+I_d{N}_f^2+I_w}{r_{eff}^2}\right\}}$$

(22)

$$a_y=\frac{1}{m_s}\left\{\mathsf{\Sigma }{F}_y+m_{s}gsin\left(x_7\right)\right\}$$

(23)

$$\dot{\psi }=x_6$$

(24)

$$\varphi =x_7$$

(25)

$$\theta =x_8$$

(26)

3.2.2. Observability Check

Before designing the estimator, the observability must be checked. The observability of the nonlinear state space model can be checked by the Lie derivative [33]. When the state space equation is expressed as $\dot{x}=f\left(x,u\right), y=h\left(x\right)$, the Lie derivative and observability matrix can be written as:

$$\begin{gathered} L_f^0=h\left(x\right) \\ {L}_f^{k+1}=\frac{\partial L_f^k}{\partial x}f=\nabla L_f^k·f \end{gathered}$$

(27)

$$O={\left[\begin{array}{c}\nabla L_f^0\\ \nabla L_f^1\\ ⋮\\ \nabla L_f^{n-1}\end{array}\right]}_{x=x_0}$$

(28)

where $O$ is the observability matrix. Using the rank of the observability matrix, the system’s observability can be seen locally. As a result of checking the rank of the observability matrix, the observability matrix has full rank in range of $v_x\ne 0$; therefore, this system is locally observable for the range except $v_x=0$.

3.2.3. Dual Extended Kalman Filter Module

Among the vehicle dynamics parameters, the cornering stiffness varies under conditions such as the vehicle load. To reduce the errors while modeling, the cornering stiffness should be estimated.

This study adopts the DEKF as an estimator for reducing the error and increasing the modeling accuracy. Figure 4 shows the DEKF scheme. The state vector, state input, state output, and state space equation have been discussed in Section 3.2.1. The DEKF module works according to the following recursive algorithm [34]:

Parameter prediction:

$${\widehat{x}}_p^{-}\left(t\right) ={\widehat{x}}_p\left(t-1\right)$$

(29)

$$P_p^{-}\left(t\right) =P_p\left(t-1\right)+Q_p$$

(30)

State prediction:

$${\widehat{x}}_s^{-}\left(t\right) =f\left({\widehat{x}}_s\left(t-1\right),u\left(t\right),{\widehat{x}}_p^{-}\left(t\right)\right)$$

(31)

$$P_s^{-}\left(t\right) =F_s\left(t\right)P_s\left(t-1\right)F_s^T\left(t\right)+Q_s$$

(32)

State update:

$$K_s\left(t\right) =P_s^{-}\left(t\right)H_s^T\left(t\right){\left[H_s\left(t\right)P_s^{-}\left(t\right)H_s^T\left(t\right)+R_s\right]}^{-1}$$

(33)

$${\widehat{x}}_s\left(t\right) ={\widehat{x}}_s^{-}\left(t\right)+K_s\left(t\right)\left[y\left(t\right)-h\left({\widehat{x}}_s^{-}\left(t\right)\right)\right]$$

(34)

$$P_s\left(t\right) =P_s^{-}\left(t\right)-K_s\left(t\right)H_s\left(t\right)P_s^{-}\left(t\right)$$

(35)

Parameter update:

$$K_p\left(t\right) =P_p^{-}\left(t\right)H_p^T\left(t\right){\left[H_p\left(t\right)P_p^{-}\left(t\right)H_p^T\left(t\right)+R_p\right]}^{-1}$$

(36)

$${\widehat{x}}_p\left(t\right) ={\widehat{x}}_p^{-}\left(t\right)+K_p\left(t\right)\left[y\left(t\right)-h\left({\widehat{x}}_s^{-}\left(t\right)\right)\right]$$

(37)

$$P_p\left(t\right) =P_p^{-}\left(t\right)-K_p\left(t\right)H_p\left(t\right)P_p^{-}\left(t\right)$$

(38)

where the parameter vector ${\widehat{x}}_p={\left[C_{\alpha f} C_{\alpha r}\right]}^T$, state vector ${\widehat{x}}_s={\left[v_x v_y v_z \dot{\varphi } \dot{\theta } \dot{\psi } \varphi \theta \right]}^T$; $P_p$ and $P_s$ are the error covariance matrices for parameters and states, respectively; $Q_p$ and $Q_s$ are the process noise covariance matrices for parameter and state estimators, respectively; $R_p$ and $R_s$ are the output noise covariance matrices for parameter and state estimators, respectively. $R_p$ and $R_s$ are the same because both the parameter and state estimator have same output y. $K_p$ and $K_s$ are the Kalman gain matrices for parameter and state estimators, respectively. $F_s$ and $H_s$ are the Jacobian matrices for the state and output equations, respectively, and are expressed as follows:

$$F_s=\left[\begin{array}{ccc}\frac{\partial f_1}{\partial x_1}& \cdots & \frac{\partial f_1}{\partial x_8}\\ ⋮& \ddots & ⋮\\ \frac{\partial f_8}{\partial x_1}& \cdots & \frac{\partial f_8}{\partial x_8}\end{array}\right]$$

(39)

$$H_s=\left[\begin{array}{ccc}\frac{\partial h_1}{\partial x_1}& \cdots & \frac{\partial h_1}{\partial x_8}\\ ⋮& \ddots & ⋮\\ \frac{\partial h_5}{\partial x_1}& \cdots & \frac{\partial h_5}{\partial x_8}\end{array}\right]$$

(40)

In Equations (36) and (38), $H_p$ is the Jacobian matrix of state output for the parameter estimator and can be expressed as:

$$H_p=\frac{\partial y}{\partial {\widehat{x}}_p}=\left[\begin{array}{c}\begin{array}{cc}\frac{\partial a_x}{\partial C_{\alpha f}}& \frac{\partial a_x}{\partial C_{\alpha r}}\end{array}\\ ⋮\\ \begin{array}{cc}\frac{\partial \theta }{\partial C_{\alpha f}}& \frac{\partial \theta }{\partial C_{\alpha r}}\end{array}\end{array}\right]$$

(41)

$R_p$ and $R_s$ can be determined by the engineer’s tuning based on sensor noise. $Q_s$ and $Q_p$ can also be determined by engineer’s tuning but $Q_p$’s square of each elements is tuned to approximately 1% of the initial values of the each actual parameter.

4. Results and Analysis

This section presents and discusses the experimental verification results of the algorithms mentioned in Section 3. Neural network and DEKF are discussed separately in Section 3.1 and Section 3.2, respectively. The performance is compared with that of commercial sensors.

4.1. Roll and Pitch Estimator (Neural Network)

4.1.1. Dataset

Sensor data from real-car experiment data were used for training and validating the neural network. The neural network input data set contained information of a car’s chassis sensors, and the label data set of roll and pitch angles was obtained from the high-accuracy GPS-inertial navigation system (INS) RT-3002 (Oxford Technical Solutions Ltd., Bicester, UK). For training the neural network, a total of 176,259 data sets were used, which logged about 30 min at 10 ms intervals in various situations and offline validation was performed in scenarios as shown in

Table 3. Roll/pitch estimator validation scenario.

	Condition	Description
Case 1	-	A stationary situation on the uphill
Case 2	Acceleration $\pm$0.5 g or higher Steering $\pm$5 deg or higher	Rapid acceleration with steering on downhill slope-quick deceleration
Case 3	Steering $\pm 50$ deg or higher Yaw rate $\pm$30 deg/s or higher	Accelerate-turn-deceleration with steering on flat road
Case 4	-	Common driving

with the same vehicle. The software used Python and the framework used TensorFlow 1.6.

4.1.2. Validation Result and Analysis

The estimation performance was validated using offline sensor data logged in various cases, as shown in Table 3. The root mean square error (RMSE) was calculated by comparison with RT-3002, which was treated as a reference, and with the datasheet of SST810, which is a commercial inclinometer sensor (Vigor Technology Co., Ltd., Xiamen, China). Figure 5, Figure 6, Figure 7 and Figure 8 show the scenario and roll/pitch estimation results for Case 1–4, respectively.

Table 4. Root mean square error (RMSE) calculation results of roll/pitch estimator and accuracy of SST810 datasheet.

	Case 1		Case 2		Case 3		Case 4
	Roll	Pitch	Roll	Pitch	Roll	Pitch	Roll	Pitch
RMSE (deg)	0.1133	0.4188	0.0573	0.5422	0.1359	0.0958	0.5140	0.5283
Commercial sensor accuracy (deg) 1	≤±0.05 (static situation)		≤±0.5 (dynamic situation)

¹ From SST810 inclinometer datasheet of Vigor Technology Co., Ltd.

shows the accuracy of the SST810 datasheet and RMSE calculation results of the estimation results for Case 1–4. Figure 5, Figure 6, Figure 7 and Figure 8 show that the value of pitch and roll angles between 0 and 2 s is fixed at zero. This is due to the structure described in Section 3.1.2, requiring 2 s of sequential data for the input; thus, the values in the initial 2 s cannot be calculated. The RMSE was therefore calculated in the time zone excluding the initial 2 s.

Case 1 shows that the roll RMSE is approximately 0.1° and the pitch RMSE is approximately 0.4°. There was some offset error, although the vehicle’s speed was zero, i.e., a stationary scenario. For commercial sensors, the error rate is 0.05° at the static scenario, but the estimation results show a larger error compared with the commercial sensors. The neural network estimator in this study uses data of only the chassis sensors of the vehicle; thus, the performance of sensors has a significant impact on the estimation performance. In particular, it can be expected that the IMU’s characteristic bias error affected the offset errors in the estimation results.

Cases 2–4 include scenarios in which rapid changes in vehicle attitude occur in areas where acceleration or deceleration occurs. The validation results show high estimation accuracy in these scenarios. In addition, the noise reduction effect compared to RT-3002 is noticeable in the 15–60 s duration for Case 3; thus, the neural network estimator can estimate stable output. The performance of the estimator is thus superior compared to the commercial sensors.

4.2. Acceleration and Angular Velocity Estimator (DEKF)

4.2.1. Validation Environment

The acceleration and angle velocity estimator was validated by simulation using CarSim, which is a commercial vehicle simulator software. To create an environment similar to the actual vehicle, Gaussian white noise was added to the sensor value, as shown in

Table 5. Sensor configuration for simulation.

Sensor	Noise	Unit
IMU ($a_x, a_y$)	0.1 (RMS) + 10 (%)	m/s2
IMU ($\dot{\psi }$)	0.01 (RMS) + 10 (%)	rad/s
Steering angle	0.05 (RMS) + 10 (%)	rad
Engine torque	7 (RMS)	N$·$m
Brake pressure	0.05 (RMS)	MPa

. To ensure that the errors of the acceleration sensor due to roll and pitch have been corrected, simulations were conducted in the scenarios shown in

Table 6. Acceleration/angular velocity estimator validation scenario.

	Condition	Description
Case 1	Roll $\pm$30 deg or lower Pitch $\pm$3 deg or lower	U-turn with 30 degrees of bank angle
Case 2	Roll $\pm$30 deg or lower Pitch $\pm 10$ deg or lower	Sharp turn at 30 degrees of bank angle after 10 degrees of uphill and downhill
Case 3	-	Common driving

.

4.2.2. Validation Results and Analysis

The correctness performance of $a_x$, $a_y$, and $\dot{\psi }$ was validated by comparing IMU sensor values, DEKF estimates, and reference values, while the estimates of roll rates and pitch rates without sensors were validated by comparing with only the reference values. The RMSE of the estimated results is calculated based on the reference and compared with the datasheet of SMI860, which is a commercial six-dimensional IMU from BOSCH Co., Ltd. In addition, to validate the effects of DEKF’s parameter estimator, the estimated results of $C_{\alpha f}$ and $C_{\alpha r}$ for each case were noted, and the estimated results with and without cornering stiffness estimation for Case 3 were compared. Figure 9, Figure 10, Figure 11 and Figure 12 show the simulation scenarios and results of DEKF.

Table 7. RMSE calculation results of DEKF and accuracy obtained from SMI860 datasheet.

		Accuracy
		DEKF (RMSE)	Commercial Sensor 1
$a_x$ (m/s2)	Case 1	0.4325	≤±0.5
	Case 2	0.8075
	Case 3	0.3087
$a_y$ (m/s2)	Case 1	0.8232	≤±0.5
	Case 2	0.4204
	Case 3	0.5085
$\dot{\psi }$ (deg/s)	Case 1	0.7391	≤±3
	Case 2	0.3953
	Case 3	0.5844
$\dot{\varphi }$ (deg/s)	Case 1	5.8499	≤±2
	Case 2	5.1394
	Case 3	1.0542
$\dot{\theta }$ (deg/s)	Case 1	1.9251	- 2
	Case 2	1.7475
	Case 3	0.6704

¹ From SMI860 IMU datasheet of BOSCH Co., Ltd. ² SMI860 cannot sense the pitch rate.

shows the accuracy of the commercial sensor, obtained from its datasheet, and RMSE calculation results of the estimation values for case 1–3. In case 1, there was a fatal error of sensor value of $a_y$ due to roll angle, and case 2 exhibited considerably varying roll and pitch angles; therefore, the value of $a_x$ and $a_y$ from sensors may not be accurate. Figure 9b and Figure 10b confirm that these errors in sensor values are successfully corrected to obtain estimates close to the reference. In addition, in normal driving scenarios, as in the 60–65 s duration in Case 3, an $a_x$ sensor error by pitch is noted, which has also been successfully corrected. Furthermore, in all cases, filter effects that reduce noise from existing sensors can be checked through $a_x$, $a_y$, and $\dot{\psi }$. Compared with a commercial sensor, $a_x$ and $a_y$ show similar accuracy, whereas $\dot{\psi }$ shows a significantly higher accuracy. However, in the case of commercial sensors, there may be errors caused by roll and pitch angles; therefore, DEKF can have commercial sensor level or higher accuracy even though correcting those errors.

The roll and pitch rates could be estimated because they were included in the state vector, although the sensor values were not included. The roll rate accuracy was occasionally lower than that of commercial sensors depending on the case, and in the case of pitch rate, it is not comparable because there is no output of commercial sensors, but it can be confirmed from the RMSE values that the estimated values have a fairly high accuracy.

Figure 9d, Figure 10d and Figure 11d show the estimated cornering stiffness. Figure 12 and

Table 8. RMSE of with and without cornering stiffness estimation in Case 3.

	RMSE
	Without Cornering Stiffness Estimation	With Cornering Stiffness Estimation
$a_y$ (m/s2)	0.5541	0.5085
$\dot{\psi }$ (deg/s)	0.6131	0.5844

show the results with and without cornering stiffness estimation in Case 3, which can improve accuracy by approximately 3–5%, particularly with a greater effect on $a_y$.

5. Conclusions and Future Work

In this paper, we proposed a CNN-based neural network to estimate the roll and pitch angles of a vehicle. A DEKF was used to correct the gravitational effect caused by the roll and pitch for estimating the exact acceleration and angular velocity.

By using the vehicle’s chassis sensor data as a time series, the neural network could estimate the roll and pitch angles of the vehicle without a GPS or six-dimensional IMU. Based on the estimated roll and pitch angels, we designed an extended Kalman filter (EKF) using the 6-DOF vehicle model. Another EKF was designed, and the two EKFs were used to estimate the cornering stiffness. We then constructed the DEKF.

Using experimental data obtained using a real car, the proposed roll and pitch estimator was validated, and the DEKF was validated in the CarSim simulation environment. The roll and pitch estimator showed an improved performance compared to the commercial sensors in dynamic scenarios and also reduced the noise. However, the performance in static scenarios was weaker. The acceleration and angular velocity estimator could effectively correct the acceleration sensor error due to roll and pitch with a de-noising effect. In addition, the roll and pitch rates that could not be obtained from sensors could be estimated with significant accuracy. By comparing the results before and after including the cornering stiffness, we found that the accuracy is improved if the cornering stiffness is considered.

On the other hand, our work has limitations and challenges that should be further discussed. We plan to consider fusion with other algorithms to improve the attitude estimation performance in static scenarios. In addition, the proposed method has not been checked in case of a change in the vehicle weight. It will also be necessary to verify the performance of the algorithm due to changes in vehicle weight. Furthermore, our proposed algorithm is hard to apply as an embedded system in vehicle because the neural network has large capacity. The future work should be conducted to enable algorithm to operate as real-time in vehicle through simplification and optimization.