Encoder–Decoder-Based Velocity Prediction Modelling for Passenger Vehicles Coupled with Driving Pattern Recognition

Abstract

To improve the performance of predictive energy management strategies for hybrid passenger vehicles, this paper proposes an Encoder–Decoder (ED)-based velocity prediction modelling system coupled with driving pattern recognition. Firstly, the driving pattern recognition (DPR) model is established by a K-means clustering algorithm and validated on test data; the driving patterns can be identified as urban, suburban, and highway. Then, by introducing the encoder–decoder structure, a DPR-ED model is designed, which enables the simultaneous input of multiple temporal features to further improve the prediction accuracy and stability. The results show that the root mean square error (RMSE) of the DPR-ED model on the validation set is 1.028 m/s for the long-time sequence prediction, which is 6.6% better than that of the multilayer perceptron (MLP) model. When the two models are applied to the test dataset, the proportion with a low error of 0.1~0.3 m/s is improved by 4% and the large-error proportion is filtered by the DPR-ED model. The DPR-ED model performs 5.2% better than the MLP model with respect to the average prediction accuracy. Meanwhile, the variance is decreased by 15.6%. This novel framework enables the processing of long-time sequences with multiple input dimensions, which improves the prediction accuracy under complicated driving patterns and enhances the generalization-related performance and robustness of the model.

1. Introduction

Energy management strategies are used to determine the power distribution between different energy sources at each moment to improve the performance of hybrid vehicles while satisfying torque requirements and are one of the key technologies of hybrid vehicles [1]. Up to now, scholars have conducted a great deal of research on energy management strategies, which are mainly divided into three major categories: rule-based, optimization-based, and learning-based [2,3,4]. Rule-based energy management strategies (RB-EMSs) are designed with suitable control rules for energy distribution according to the characteristics of the controlled object. The process of design depends mainly on the designers’ knowledge and continuous exploration and experimentation. Moreover, the differences in the control rules between different models require repeated calibrations and cause difficulty with respect to obtaining optimal results. RB-EMSs are widely used due to their simple logic, easy implementation, and high stability [5]. As its research progressed, the addition of fuzzy logic improved the performance of RB-EMSs [6]; however, on the one hand, researchers continue to pursue higher requirements for all aspects of hybrid vehicle performance, and on the other hand, the disadvantages of RB-EMS will gradually appear due to its inherent poor portability, cumbersome tuning of parameters, and inability to achieve optimal control. Optimization-based energy management strategies use optimization algorithms for energy management, which are less empirically dependent than RB-EMSs and can achieve better results. Instantaneous optimization algorithms are those algorithms that do not need to know all the path-related information and can be optimized using only the current information, including the equivalent fuel consumption minimization strategy (ECMS) [7,8] and adaptive equivalent fuel consumption minimization strategy (A-ECMS) [9,10]. The optimization effect of both ECMS and A-ECMS are closely related to the equivalence factor and often have some distance from the global optimal solution. The global optimization algorithm represented by dynamic programming (DP) can obtain a global optimal solution, but at the same time, DP needs to anticipate the vehicle-state information of the entire future driving cycle in advance. The optimal strategy is obtained only for that particular cycle. Therefore, given the complex and variable driving conditions in the real world, such energy management strategies cannot be applied in practice and are often used as references for evaluating and optimizing other energy management strategies [11,12]. With the development of energy management, algorithms such as model predictive control (MPC) obtain locally optimal solutions by continuous roll-forward optimization within the prediction-sight distance, which are neither short-sighted nor sensitive compared to instantaneous optimization algorithms. An advanced knowledge of all the future driving information is not required compared to global optimization algorithms, so it has more potential for real-vehicle applications [13]. With the development of artificial intelligence technology, learning-based algorithms based on Reinforcement Learning (RL) have gained widespread attention, as they can learn energy management criteria from training samples [14], be applied to different driving conditions [15], adapt to different prediction sight distances [16], require no discretization of state nor control parameters [17], and provide the possibility of finding a theoretical global optimal solution [18]. In summary, short-term predictive energy management strategies that obtain locally optimal solutions by rolling the prediction sight distance are the focus of the current research. Among them, the short-term vehicle velocity prediction algorithm is the top priority of the predictive energy management strategies, which greatly affects the performance of these strategies [19,20], and the velocity prediction algorithm also plays a crucial role in the fields of traffic flow prediction and active vehicle safety control [21].

The mainstream of short-term vehicle velocity prediction methods is divided into stochastic and deterministic vehicle velocity prediction algorithms [22]. The Markov chain-based vehicle velocity prediction algorithm is the most representative stochastic vehicle velocity prediction algorithm [23,24]. Although the accuracy of vehicle velocity prediction can be improved by establishing a multi-level Markov chain model, it leads to an exponential expansion of the size of the probability matrix, which leads to an exponential rise in the computation time of the algorithm, and even so, it is still difficult to cover all possible Markov states [25]. Deterministic velocity prediction algorithms can be further divided into parametric and non-parametric velocity prediction algorithms. A parametric vehicle velocity prediction algorithm refers to the prediction by establishing a model with parameters, and the typical case is to establish an auto-regressive moving average model (ARMA) for a time series analysis, which is proven to have a wide range of engineering-control application cases [26]. However, for vehicles driving on roads, the randomness of the travel path and the lack of information sharing of individual vehicles leads to limitations on the ability of parametric vehicle velocity prediction models, causing larger prediction errors than non-parametric vehicle velocity prediction algorithms [27]. Instead, non-parametric velocity prediction algorithms use historical data to build a prediction model for future velocity prediction, so they are also called data-driven velocity prediction algorithms and currently represent a growing trend [28]. The neural network model is a typical data-driven algorithm due to its powerful nonlinear mapping capability and strong robustness. It has become one of the key technologies in the field of predictive modeling and optimal control of nonlinear systems [29] and is also widely applied to the field of vehicle velocity prediction. Scholars have conducted in-depth research on neural network-based vehicle velocity prediction methods and fruitful results have been achieved. Artificial neural networks (ANN) [16], long short-term memory neural networks (LSTM) [30], nonlinear autoregressive neural networks with additional inputs (NARX) [31], and RBFNN [32] are several neural networks that have been mostly studied in the field of vehicle velocity prediction. Among them, LSTM networks are widely used in time series prediction models and can maintain the internal input memory and compensate for the gradient vanishing and gradient explosion that exist in RNN during the training process [33]. In the field of vehicle velocity prediction, LSTM networks are currently used as the “Encoder” structure with a multi-step input and single-step output, and the “Decoder” structure with a single-step input and multi-step output. The former can achieve the effect of sequence prediction by combining the output as a new input, but the velocity of computation needs to be improved; the latter can achieve sequence prediction directly, but the structure lacks memory for the information of long history sequences.

Up to now, there are still few relevant studies on the sequence-to-sequence vehicle velocity prediction model established by the encoder-decoder framework. Nevertheless, with the continuous development of vehicle-networking technology, the information interaction data provided by vehicle networking can also be used jointly with neural networks in the future [34], and the relevant information used as the input of neural networks can also improve the vehicle velocity prediction accuracy and robustness. Therefore, it is necessary to provide a solution for the encoder–decoder-based time series prediction models with different input and output dimensions. In this paper, an innovative encoder–decoder-based velocity prediction modelling system for passenger vehicles coupled with driving pattern recognition is proposed. The model uses the encoder–decoder framework with LSTM units as neurons to establish a multi-input sequence-to-sequence model based on the characteristics of vehicle velocity sequences and the results of driving pattern recognition. Moreover, with the continuous development of the Internet of vehicles, an increasing amount of time-domain information can be accessed by vehicles. The multi-input feature of this model can improve the efficacy of velocity prediction by rearranging the input samples and retraining them as a “skeleton”, which is very rare in the existing similar model. Section 2 introduces the establishment of the method of the driving-pattern-recognition model and recognition performance as the foundation for the following velocity prediction model. Section 3 proposes the velocity prediction based on the Encoder–decoder model, including the revised structure under the different dimensions of input and output sequences and the training method. The effects of the history window, prediction window, and number of neurons on the model performance are analyzed in Section 4, and the performance of the optimal model is compared with that of the traditional neural network. Finally, in Section 5, conclusions are presented.

2. Driving Pattern Recognition

This section will take the accuracy of recognition, the real-time calculation speed, and the compilation feasibility into consideration and find a method that can use the current and historical driving information for driving pattern recognition. In this section, the K-means clustering algorithm is used for driving pattern recognition. Firstly, the training matrix is obtained with standard driving cycles, and the K-means clustering algorithm model is trained. Then, the model is used to recognize the driving patterns in the test data and the results are shown.

2.1. Dataset and Characteristics

One of the keys to recognizing driving patterns is to find a set of features that can represent certain characteristics of driving pattern information, such as the average velocity of the past period of time, idle ratio, vehicle acceleration, etc., and they are used as the basis for recognizing the current driving patterns of the vehicle. In recent years, the selection of features to reflect the characteristics of driving patterns, which can then be applied to the process of driving pattern recognition, has been a hot spot of research, in which the difficulty lies in the appropriate number and type of features to reflect the characteristics of driving patterns. As far as the number of features is concerned, selecting too many features or combining them with the method of principal component analysis for driving pattern recognition may lead to an increase in the difficulty of real-time online analysis, whereas selecting too few features will not accurately reflect the characteristics of driving patterns and lead to the poor accuracy of driving pattern recognition [35]. Hu et al. selected four features—to name three: the average velocity, idle ratio, and cruising time ratio—for driving pattern recognition, which ensured the accuracy of the model and greatly improved the computational real time [36].

In view of the above research results, three features have been selected in this paper to balance the real-time properties, accuracy, and compilation feasibility of driving pattern recognition, including the average velocity (v_mean), maximum velocity (v_max), and idle ratio (T_idle).

After the selection of the features, typical driving cycles need to be selected as the training set; six standard driving cycles are selected in this paper. The features of each standard driving cycle are calculated separately and used to train the K-means clustering model, so that the model can realize the real-time recognition of any driving patterns in actual driving. Standard driving cycles are mostly designed for the driving pattern s in their respective regions. In order to enhance the adaptability of the traffic conditions in different regions, this paper takes into full consideration the representative driving cycles of various regions in the world when selecting the typical driving cycles so that the regional limitations of the recognition on driving patterns is reduced. As a result, this paper refers to six representative standard driving cycles, including the WLTC, which is the most commonly used in the world; the NEDC introduced by the European Union; the CLTC-P introduced by China; and the Artemis cycles introduced by the United States. The velocity versus time of the standard driving cycles selected in this paper are shown in Figure 1.

After the typical driving cycles are selected, the division method of the driving cycle samples needs to be determined. The aim of the driving cycle division is to build a sample database, which is the premise of the driving pattern clustering analysis. In the existing work, there are two methods used to divide the driving cycle samples: one is the division method based on a certain time window, and the other is the division method based on micro-trip driving cycle samples. In this paper, we adopt the division method based on a certain time window, as shown in Figure 2. When the time window is determined (the length of the time window is selected as 60 s in this paper), the velocity sequence is intercepted every 60 s as a driving cycle sample from the starting point to the end.

2.2. K-Means Cluster Analysis

A K-means algorithm is applicable to large sample data for clustering and enabling a more reasonable classification. The optimization of K-Means entails minimizing the within-cluster sum of squared errors (SSE), as shown in Formula (1):

$$SSE={\displaystyle \sum }_{m=1}^k{\displaystyle \sum }_{x\in C_i}dist{\left(X,c_i\right)}^2={\displaystyle \sum }_{m=1}^k{\displaystyle \sum }_{x\in C_i}\Vert X,c_i\Vert$$

(1)

where $dist\left(X,c_i\right)$ is the distance between object X and the clustering center c_i, and dist denotes the Euclidean distance between any two objects in D.

The SSE can be analyzed by the elbow rule. The SSE decreases with the increase in the number of clustering results, but for data with a certain degree of differentiation, the SSE improves greatly when a certain point is reached and decreases slowly afterwards, and this point can be considered as the point with the best clustering performance.

For a reasonable evaluation and analysis of the clustering performance, the Silhouette Coefficient (SC) evaluation method is also a commonly used evaluation method [37]. Assuming that the average distance of the objects in the clusters containing data x and x’ is defined as a(x), b(x) is the minimum average distance from the cluster containing x to all the clusters that do not contain x′. a(x), b(x), and the Silhouette coefficient S(i) are defined as follows:

$$a\left(x\right)=\frac{{{\displaystyle \sum }}_{x^{\prime }\in C_i,x\ne x^{\prime }}\Vert x-x^{\prime }\Vert }{\left|C_i-1\right|}$$

(2)

$$b\left(x\right)=\underset{C_j:1\le i\le K,x\ne x^{\prime }}{\min }\frac{{{\displaystyle \sum }}_{x^{\prime }\in C_j}\Vert x-x^{\prime }\Vert }{\left|C_j\right|}$$

(3)

$$S\left(i\right)=\frac{b\left(i\right)-a\left(i\right)}{\max \left\{b\left(i\right)-a\left(i\right)\right\}}$$

(4)

where a(i) denotes the maximum distance of the sample i from other samples in the same cluster, and b(i) denotes the minimum distance of the sample i from all samples in the other clusters.

The optimal number of clustering centers can be determined based on S(i). The closer S(i) is to 1, the more reasonable the sample clustering result; the closer S(i) is to −1, the less accurate the sample clustering result; if S(i) is close to 0, the sample is at the boundary position between two clustering results.

Combining the SC evaluation method and the elbow rule of clustering, the SSE and S(i) versus the number of driving pattern clustering centers are calculated, respectively, as shown in Figure 3.

It is evident from Figure 3 that the changing rate of SSE is greatly decreased when the number of clustering, k, is taken as 3 or 4, so k = 3 or k = 4 can be considered as the number of clustering; the Silhouette coefficient is 0.55 when k is taken as 3 and 0.50 when k is taken as 4. So, the clustering result is more reasonable when k is taken as 3 by the SC evaluation method. As a result, the number of clustering selected in this paper is 3. Combined with the actual driving scenarios, these 3 patterns are defined as urban, suburban, and highway.

The clustering results are shown in Figure 4 on training data and under the three-dimensional coordinates of the average velocity, idle ratio, and maximum velocity.

Figure 4a shows the prediction results from standard cycles as training data. The driving-pattern-recognition model can match the patterns represented by the different time axes of each standard cycle. Further analysis of the clustering results in Figure 4b shows that the K-means clustering center of red points is (18.10, 0.31, 38.00), which has a low average velocity, a high idle ratio, and a low maximum velocity, so the red points represent a congested urban driving condition. The K-means clustering center of the blue points is (100.60, 0, 111.94); the average velocity of this cluster is high, the idle ratio is 0, and the maximum velocity is also high, so the blue points represent smooth highway-driving conditions. The K-means clustering center of the green points is (48.80, 0.04, 65.74), and the average velocity, idle ratio, and maximum velocity of this cluster are between the congested urban driving condition and the smooth highway driving condition. As a result, these points represent the suburban driving conditions. Thus, the driving conditions can be broadly classified into three typical patterns, namely, urban, suburban, and highway. The classification results are consistent with the actual driving conditions mentioned above, so the recognition results of the model can be considered reasonable.

2.3. Driving Pattern Recognition

Since only three features are used for driving pattern recognition in this paper, after ensuring the real-time model calculation and the low compilation difficulty of the algorithm, in order to further illustrate the accuracy of the driving pattern recognition, actual road spectrum data are collected as a test dataset to verify the recognition accuracy of the model based on the K-means clustering algorithm. The driving-pattern-recognition model will calculate the features in real time and output the recognition results based on the historical 60 s data during the driving process. Due to some abnormal data on velocity caused by the driver’s operation and GPS device in the actual road spectrum data (as is the case for many real-world applications), the impulse noise will affect the characteristic of v_max, which may cause an instance of incorrect recognition by the model. When applying the driving-pattern-recognition model to practical predictions, the measured velocity signal should be smoothed using the moving average-filtering method to reduce the random impulse noise [38]. As shown in Formula (5), the basic idea of the moving average filtering method is to perform local averaging on a small number of points along the data of length N, thereby filtering out random noise.

$$y_f\left(k\right)=\frac{1}{m}{{\displaystyle \sum }}_{k-\left(m-1\right)/2}^{k+\left(m-1\right)/2}y\left(i\right)$$

(5)

where $y\left(·\right)$ represents the original data; $y_f\left(·\right)$ represents the filtered data; m is the number of data points used for filtering, which is odd; and i and k represent the data point number. In this paper, m is selected as 5, which represents a 5-second moving average.

Figure 5 represents the prediction results of the driving-pattern-recognition model on the filtered test data.

As shown in Figure 5, the total length of the actual road spectrum data is about 16,000 s, which includes an about 3500 s length of the urban pattern, a 6250 s length of the suburban pattern, and a 6250 s length of highway pattern. Three patterns are shown in the figure as driving patterns one, two, and three, which are indicated by the blue dashed line. The red solid line indicates the prediction result of the driving-pattern-recognition model based on the K-means algorithm. The comparison shows that the driving-pattern-recognition model can identify urban patterns completely and accurately, and the recognition accuracy rate reaches 100%. The suburban pattern is more complicated due to the intermingling of vehicles driving at medium and low velocity or even idling, so the model identifies some of the segments with lower average velocities (e.g., 9000~9100 s segments) or longer idling ratios (e.g., 9600~9800 s segments) as an urban pattern. The prediction result of the highway patterns is also accurate, among which the velocity in the segment 12,750–12,950 s has a sudden drop, which may be due to unexpected traffic accidents or other unexpected situations in the highway pattern at the time of driving, so it cannot be used as a typical feature of a highway pattern; instead, it is more reasonable for it to be identified as a suburban or urban (congested) pattern. In summary, the driving-pattern-recognition model in this paper ensures the real-time calculation speed and low compilation difficulty of the algorithm, while its recognition accuracy is satisfactory, reaching 84.1%, which can be used as the basis for subsequent research.

3. Velocity Prediction Based on the Encoder–Decoder Model

3.1. Basic Conception of LSTM

The LSTM network is an improvement of RNN with a unique store-and-forget function compared to the traditional RNN. By learning the sequence input and extracting the hidden sequence features, it can obtain the dependency relationship between sequences accurately, and overcome the complications of gradient vanishing, gradient explosion, and long-term memory disappearance that occur in RNN during training [39].

There are three inputs to the LSTM network at moment t: the input value x_t at the current moment, the output value h_t−1 at the previous moment, and the cell state c_t−1. The input gate i_t, the output gate o_t, and the forget gate f_t receive the same inputs [h_t−1, x_t], which are used to control the update process of the cell state c_t after the activation function σ. The basic LSTM network is shown in Figure 6.

A gate is a fully connected layer that receives a vector as an input, while the output is a vector of real numbers between 0 and 1. W is the weight matrix of the gate and b is the bias; then, the gate can be represented as:

$$g\left(x\right)=\sigma \left(Wx+b\right)$$

(6)

where σ denotes the activation function. In this paper, the Sigmoid function is used as the activation function to adjust the input [h_t−1, x_t] to the (0,1) interval.

The input gate i_t controls how much of the input x_t is saved to the cell state c_t at the current moment, and is defined as:

$$i_t=\sigma \left(W_i\left[h_{t-1},x_t\right]+b_i\right)$$

(7)

The forget gate f_t controls how much of the cell state c_t−1 of the previous moment is saved to the current moment state c_t, and is defined as:

$$f_t=\sigma \left(W_f\left[h_{t-1},x_t\right]+b_i\right)$$

(8)

The output gate o_t controls how much of the state c_t at the current moment is output to the current output value h_t, and is defined as:

$$o_t=\sigma \left(W_o\left[h_{t-1},x_t\right]+b_i\right)$$

(9)

The cell state ${\tilde{c}}_t$ of the current input is described according to h_t−1 and the current input x_t, and is defined as:

$${\tilde{c}}_t=tanh\left(W_c\left[h_{t-1},x_t\right]+b_c\right)$$

(10)

The cell state c_t is adjusted by the input gate i_t and the forget gate f_t and is defined as:

$$c_t=f_t{c}_{t-1}+i_t{\tilde{c}}_t$$

(11)

The final output h_t of the network is determined by both the output gate o_t and the cell state c_t and is defined as:

$$h_t=o_{t}tanh\left(c_t\right)$$

(12)

In Formulas (7)–(10), the matrices W_i, W_f, W_o, and W_c are the gate weight matrices and the vectors b_i, b_f, b_o, and b_c are the bias terms of the gates.

The information used for updating the cell state c_t, f_t, and o_t are determined by the gating vectors in Formulas (7)–(10). The cell state and output are updated by Formulas (11) and (12). The cell state is reset or restored by f_t and the new state c_t is obtained by adding partial information through the input gate i_t, while the hidden state h_t is controlled and updated by the output gate o_t.

3.2. An Encoder–Decoder Structure Coupled with Driving Pattern Recognition

The encoder–decoder framework is a structure for time series analysis. In this paper, LSTM neurons are used for establishing the encoder–decoder model and coupled with the driving pattern recognition results obtained above. This multi-input model inputs both the vehicle velocity and driving pattern recognition sequences and the constructed input sequences are encoded and decoded to achieve vehicle velocity prediction, as shown in Figure 7. The driving pattern recognition-encoder decoder (DPR-ED) model mainly contains two parts: the encoder side and the decoder side.

The operation of the ED-based model consists of two phases: training and testing. The training phase is for learning the implied knowledge and experience from the training data, and the weight matrix of the model is trained for the testing phase; the testing phase applies the trained model and the input data for testing to calculate the corresponding output data. The training and prediction samples are the preparatory work for running the model. All the above-mentioned training and prediction datasets of the DPR-ED model are closely related to the parameters of the history window (W_t), which represents the length of the history series, and the prediction window (W_n), which represents the length of the prediction series. For the traditional ED framework shown in Figure 7a, the input of the training data is a $W_t\times M$ matrix and the output of the training data is a $W_n\times M$ matrix; the input of the prediction data is a $W_t\times 1$ vector and the output of the prediction data is a $W_n\times 1$ vector, where M is the number of training samples. For the DPR-ED model proposed in this paper, shown in Figure 7b, the training data input is a $C\times W_t\times M$ tensor and the training data output is a $W_n\times M$ matrix; the prediction data input is a $C\times W_t$ matrix and the prediction data output is a $W_n\times 1$ vector, where C is the number of features. The number of features in this paper is two, namely, vehicle velocity and driving pattern.

The operation process of the DPR-ED model is as follows. Using the time window sliding method, M training input samples are intercepted from the sequence of typical driving cycles and the corresponding driving pattern sequence, each of which is in the form of $\left[\begin{array}{c}{v}_1 \\ r_1\end{array}\begin{array}{c}{v}_2 \\ r_2\end{array}\begin{array}{c}\dots \\ \dots \end{array}\begin{array}{c}{v}_t \\ r_t\end{array}\right]$, forming the training input data tensor ${\mathit{X}}_{\mathit{t}\mathit{r}\mathit{a}\mathit{i}\mathit{n}}=2\times W_t\times M$. According to the same method, M training output samples are intercepted from the sequence of typical driving cycles and the corresponding driving pattern sequence. Each of them is in the form of $\left[v_{t+1} v_{t+2}\dots v_{t+n}\right]$, forming the training output data matrix ${\mathit{Y}}_{\mathit{t}\mathit{r}\mathit{a}\mathit{i}\mathit{n}}=W_n\times M$. The number of training samples obtained by the time window-sliding method in this paper is 7914, that is, M = 7914. Similarly, the validation input data tensor ${\mathit{X}}_{\mathit{v}\mathit{a}\mathit{l}}=2\times W_t\times N$ and the validation output data matrix ${\mathit{Y}}_{\mathit{v}\mathit{a}\mathit{l}}=W_n\times N$ are obtained. In this paper, the time window-sliding method is used to randomly select one-tenth of the actual road spectrum data as the validation set. Such a small amount of extraction does not affect the persuasiveness of the model analysis on the test set, and even improves the performance of the model by using the complete input of typical driving cycles as training. The number of validation samples in this paper is 1594, that is, N = 1594. After the model is trained, the test data input matrix ${\mathit{X}}_{\mathit{t}\mathit{e}\mathit{s}\mathit{t}}=2\times W_t$ is intercepted from the test dataset using the time window-sliding method. The prediction result will be calculated by the trained model as the data output vector ${\mathit{Y}}_{\mathit{t}\mathit{e}\mathit{s}\mathit{t}}=W_n\times 1$, which is the predicted vehicle velocity sequence.

In terms of the structure of the model, the number of layers of the DPR-ED model in this paper is fixed to a single layer. Setting the number of nodes in the input layer—which represents the length of the historical velocity sequence, W_t—too small may decrease prediction accuracy for a lack of historical information, while setting it too large may increase the complexity of the model structure and thus decrease the computational speed. Regarding the number of nodes in the output layer, which represents the length of the predicted vehicle velocity sequence, W_n, it is indicated in the literature [22] that a prediction sight distance between 1~10 s is beneficial for the PEMS effect of HEVs, while [40] determines it as 5 s and obtains the best results. Considering the above, in the subsequent study of this paper, the range of W_n is set as 1~5 s, and the range of W_t is set as twice the length of W_n, which is 2~10 s. A detailed parameter study and optimization of W_n and W_t are carried out in the subsequent section. The number of neurons in the hidden layer determines the performance of the neural network. Too few neurons will result in the model’s inability to obtain enough fitting features and too many will cause the model to run slower and be prone to overfitting, so both conditions will lead to poor prediction results of the trained neural network. The number of hidden-layer neurons is also one of the important parameters to be studied and optimized in detail in this paper.

In order to evaluate the prediction accuracy, different evaluation methods have been used in the literature, the most commonly used of which is the root mean square error (RMSE), which reflects the error between the predicted vehicle velocity and the actual vehicle velocity at each step of the predicted sight distance, as shown in Formula (13):

$$RMSE=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{k=1}^n{\left(Y_P^k-Y_A^k\right)}^2}$$

(13)

where n is the length of predicted sight distance, which represents the length of the predicted vehicle velocity. It is equal to the number of output nodes of the DPR-ED model. $Y_P^k$ is the predicted vehicle velocity value corresponding to the kth predicted sight distance ($k\in \left[1,n\right]$);$Y_A^k$ is the actual vehicle velocity value corresponding to the kth predicted sight distance point.

Since the RMSE can only reflect the prediction accuracy at each forecasting step and cannot be used to evaluate the average prediction accuracy of the whole forecasting process, the introduction of an average root mean square error (RMSE_A) is proposed as shown in Formula (14):

$$RMS{E}_A=\frac{1}{M}{{\displaystyle \sum }}_{step=1}^{M}RMSE$$

(14)

where M is the number of samples. For the sake of the conciseness of expression, the RMSE is used to denote the average root mean square error in all the subsequent research in this paper.

In this paper, to manifest the performance enhancement effect of the DPR-ED model, the traditional MLP model is also established for comparison. The specific method can be found in the literature [41] and the method for comparison draws on the experience of [42].

4. Results and Discussion

Firstly, in order to obtain the optimal model performance, the relationship between the number of neurons of the traditional MLP model and the number of LSTM neurons of the ED-based models versus the fitting accuracy needs to be investigated. In the training process of the models, both the training and validation datasets were normalized. The RMSE was used as the loss. The Adam algorithm was used to update the neural network weights and the early stopping mechanism was introduced to optimize the epochs of training to prevent the overfitting of the model. The parameter of early stopping was defined as patience = 10, which means the training is stopped when the performance of the loss in the validation set is not further improved after ten consecutive weight updates. Finally, in order to better evaluate the compatibility, the accuracy and stability of the model with respect to the parameter length, the history window, and the prediction window were selected as the maximum values (10 s and 5 s) in this study, so that the trained model would have better compatibility with simpler cases. Each model was randomly assigned initial weights, which was repeated five times during training, and the results were analyzed by box plots. The influences of the different numbers of neurons on the learning performance of the traditional MLP model, Basic ED model, and DPR-ED model for the validation set are shown in Figure 8.

Figure 8a shows the accuracy of the traditional MLP model versus different numbers of hidden-layer neurons in the validation set, where each neuron is a perceptron. The mean and standard deviation of the RMSE of the MLP model in the validation set show a trend of decreasing and then increasing as the number of hidden-layer neurons of the MLP model increases, and reach the minimum values of 0.000981 and 0.000021, respectively, when the number of hidden-layer neurons is selected as 30. The accuracy of the basic ED model and DPR-ED model versus different numbers of hidden-layer neurons in the validation set are shown in Figure 8b,c. The neurons in both models are LSTM units. The result indicates that when the number of hidden-layer neurons is taken as 10 for the basic ED model and less than 20 for the DPR-ED model, the performance of the models still cannot be improved (poor stability and accuracy) even after the iterations are completed, which is due to the fact that a simple network is unable to present the characteristics of the historical velocity and the impact of historical information on the future velocity. As the number of hidden-layer neurons increase, the model performance approaches excellence and stability. Therefore, the optimal numbers of hidden-layer neurons of the basic ED model and DPR-ED model are 20 and 30, respectively. The mean values of the RMSE are 0.000872 and 0.000862 and the standard deviations of the RMSE reach 0.000023 and 0.000032, respectively.

Through the above analysis, the structure of the traditional MLP model is determined as a network with 30 hidden-layer neurons in a single layer, and the structure of the basic ED model and the DPR-ED model is determined as a network with 20 and 30 hidden-layer neurons (LSTM units) in a single layer in the subsequent study. The reason why the DPR-ED model has more hidden-layer neurons than the Basic ED model is probably because the DPR-ED model accepts a multi series input and therefore requires more LSTM units to accurately render the corresponding features.

Figure 9 shows the accuracy of each model versus the history window and prediction window on the test set and comparison.

Figure 9a,b show the impact of different history and prediction windows on the accuracy of the traditional MLP model and the DPR-ED model on the test set. From the results of these two figures, it can be clearly seen that both the history and prediction window directly affect the prediction accuracy of the model. For the MLP model, the history window has a slight effect on the prediction accuracy of the model (as seen from the equipotential line); that is, the larger the history window for the same prediction window case, the more the RMSE decreases. For the DPR-ED model, the change of the history window has little effect on the prediction accuracy of the model, so the input sequence length of the model is also less sensitive. The prediction accuracy decreases significantly as the prediction window increases, so in the process of its application in a real vehicle, an excessively large prediction window is generally not used. In addition, it is obvious from the figure that the accuracy of the DPR-ED model is higher than that of the MLP model to different degrees under the same value of the history window and the prediction window; the performance of the DPR-ED model is slightly higher than that of the MLP model when the prediction window is small (e.g., 2 s) For instance, the RMSE of the MLP model at a history window of 6 s is 0.483 m/s, while the RMSE of the DPR-ED model is 0.476 m/s, which demonstrates a slight improvement of 1.4%. When both the history and prediction window become larger, the DPR-ED model shows a greater advantage: the RMSE of the DPR-ED model is 1.028 m/s while the RMSE of the MLP model is 1.096 m/s at the history and prediction windows of 10 s and 5 s, respectively, which is a 6.6% deterioration in performance compared to the former. Figure 9c shows the performance improvement of the DPR-ED model compared to the basic ED model in the case of long prediction sequences. In addition to the role of the ED frame, which can better handle time series problems, when the prediction window is large, the LSTM neurons of the DPR-ED model receiving multidimensional inputs are also able to parse more complex information, so the accuracy is further improved by about 2.5% compared to the basic ED model.

Figure 10 shows the prediction results of the DPR-ED model from the test set. Since the total simulation time is too long and too many data points are not conducive to graphical analysis, the results of the urban, suburban, and highway patterns are intercepted to 1000 s each. In order to analyze the prediction performance for the five steps more clearly, the local velocity prediction trajectories for two random time points with different driving pattern characteristics are compared in Figure 11.

The red curve in Figure 10 represents the velocity of the five future-prediction steps, and the blue curve represents the actual velocity data. Under the different driving patterns, the DPR-ED model can basically respond to the transient process and adjust the predicted velocity within 5 s, which shows that the DPR-ED model proposed in this paper is effective. Figure 11a shows the typical characteristics of the urban driving pattern and its prediction results. It is shown from Figure 11a that the urban driving pattern contains frequent accelerations and decelerations, and the predicted velocity is close to the actual velocity when the vehicle experiences low deceleration or a smooth large deceleration at 767 s and 774 s. If a sudden change in acceleration or deceleration occurs (reflecting the various complex road conditions that occur in the urban driving pattern during actual driving), (i.e., 757 s), the predicted velocity deviates from the actual velocity, but still follows the trajectory of the trend. Figure 11b shows the typical characteristics of the highway driving pattern and its prediction results. Figure 11b indicates that the highway driving pattern shows a relatively gentle high-speed cruise or small speed changes at most of the time. The predicted velocity of the DPR-ED model also shows a relatively gentle change trend in the highway driving pattern, which is close to the actual speed. It is evident that since the DPR-ED model additionally accepts the driving pattern sequence information, it can better predict the typical characteristics under different driving patterns.

It is worth noting that even though the predicted velocity corresponds well to the actual velocity, the predicted velocity seems quite noisy, especially in the city pattern. Firstly, the unfiltered velocity could be one of the reasons for this phenomenon, as the impulse noise of the data itself will definitely affect the prediction result. Figure 12 shows the prediction results of the DPR-ED model on the city part of the filtered test set. The filtering method is five-second moving average, which has been mentioned above. Secondly, the ability to change the speed of the applied vehicle should be examined and limited while the torque limitation exists for the power system of a certain vehicle. Figure 13 shows the acceleration distribution of the prediction results by the DPR-ED model on the raw test set.

As shown in Figure 12, after filtering the impulse and high frequency noise in the raw test data, the velocity prediction stability is improved significantly in the area where the sudden change is large. The RMSE of the prediction results from the filtered test set is an exceptional 0.79 m/s. It indicates that a low-noise vehicle speed signal plays a crucial role in the efficient operation of the model. Achieving signal noise reduction is not within the scope of this paper, so the following analysis is based on raw data. However, for real-world applications, high-precision GPS and advanced signal-processing techniques should be coupled as much as possible. Figure 13 shows that the acceleration distribution is an approximate normal distribution, while the acceleration proportion is slightly more than the deceleration proportion. A small number of data points exhibit a changing rate of velocity greater than 1 m/s, and there are individual points with a changing rate greater than 1.5 m/s, which may also add to the noise of the model. When the power limit or driving style does not allow for such an acceleration, a clipping method can be applied during the real-world application of the model to correct the prediction results within the allowed acceleration range.

Figure 14 shows the local RMSE of the MLP model and the DPR-ED model during the process of the velocity prediction for the whole test set.

From Figure 14, it is evident that the local RMSE has the lowest average value in the highway driving pattern and the highest average value in the suburban driving pattern, which is due to the fact that the velocity changes in the highway driving pattern are smoother, while the complex suburban driving pattern is full of predictive uncertainties. The local RMSE is larger when the vehicle is operating under a large rate of change in acceleration, which is due to the fact that large prediction errors usually occur when the vehicle suddenly starts to accelerate after deceleration or in a situation of emergency deceleration after acceleration; such driving conditions are challenging to predict and should be addressed with the involvement of the driver. In addition, it can be found that the DPR-ED model improves the accuracy compared to the MLP model when the vehicles are working in a large rate of change of acceleration, although the local RMSE shows a peak (especially for the highway driving pattern). For example, the local RMSE of the MLP model’s predicted velocities at times 13,538 s and 14,805 s are 5.21 m/s and 5.05 m/s, respectively, while the results predicted by the DPR-ED model at the same time are 4.88 m/s and 4.61 m/s, respectively, which shows an improvement of 6.3% and 8.7%.

Figure 15 shows the comparison of the error distribution between the MLP model and the DPR-ED model for the test data.

Table 1. The proportion of the prediction results of the test set contained in each error segment.

RMSE Segment (m/s)	Proportion (MLP Model)	Proportion (DPR-ED Model)
0~0.1	12.81%	12.15%
0.1~0.3	19.81%	23.55%
0.3~0.5	18.15%	18.13%
0.5~0.7	13.22%	12.34%
0.7~1.1	14.85%	14.5%
1.1~2.1	15.35%	14.37%
>2.1	5.81%	4.96%

shows the proportion of the prediction results for the test set contained in each error segment.

As shown in Table 1, the results predicted by the DPR-ED model and the MLP model from the test data with an error of 0~0.1 m/s have similar proportions. However, the results predicted by the DPR-ED model from the test data with an error of 0.1~0.3 m/s have a proportion of 23.55%, which is about 4% higher than the result obtained by the MLP model. Therefore, more prediction results are concentrated on segments with small errors. On the other hand, the DPR-ED model has also eliminated the data with an error of more than 0.5 m/s. It reduced the proportion with an error of 0.5~0.7 m/s and over 1.1 m/s by about 1% and 1.9%, respectively; thus, the DPR-ED model filters out more of the results with larger errors. As a result, the DPR-ED model shows an excellent prediction robustness over different driving patterns.

Consequently, compared with the MLP model’s poor prediction accuracy on certain loads due to the discrete characteristics of the training data load, the optimal partially shared network shows an excellent prediction robustness over the full load range.

As shown in Figure 15, it can be found that the average error of the MLP model and the DPR-ED model prediction result is 0.81 m/s and 0.77 m/s, respectively. The DPR-ED model performs 5.2% better than the MLP model on the average prediction accuracy. Moreover, the variance is decreased greatly from 0.52 to 0.45, which is an improvement of 15.6%. This result shows that the DPR-ED model significantly improves both the accuracy and the stability.

5. Conclusions

In this paper, based on the samples from typical driving cycles, a high-precision velocity prediction model was established through the encoder–decoder framework coupled with driving pattern recognition. The main achievements and conclusions are as follows:

(1)

The driving-pattern-recognition model was established by a K-means clustering algorithm and validated based on the test data; the driving patterns were identified as urban, suburban, and highway patterns. The model achieved a satisfactory recognition accuracy of 84.1% on the total length of 16,000 s of real road spectrum data, achieving results that can be used as the basis for subsequent studies.

(2)

The MLP, basic ED, and DPR-ED models, trained using the early stopping method, were developed. The effect of different numbers of neurons on the prediction accuracy and stability of each model was investigated and the optimization of the models was completed. The results show that the DPR-ED model with 30 LSTM hidden neurons can achieve the optimal overall performance for velocity prediction, which obtains an average RMSE of 0.000862 and a standard deviation of 0.000032 after the dataset’s normalization.

(3)

Compared with the MLP model, the DPR-ED model is designed to improve the performance by implementing multidimensional inputs and applying time series analysis. In the long-time prediction series case, the DPR-ED model shows a significant advantage over the MLP model: the RMSE of the DPR-ED model applied to the validation set was 1.028 m/s, while the RMSE of the MLP model was 1.096 m/s, with a 6.6% deterioration in performance compared to the former. When the two models were applied to 16,000 s road spectrum data for testing, the proportion with a low error of 0.1~0.3 m/s was improved by 4% and a larger error proportion was filtered for the results predicted by the DPR-ED model. The DPR-ED model performed 5.2% better than the MLP model with respect to the average prediction accuracy. Meanwhile, the variance was decreased by 15.6%. This novel framework enables the processing of long-time sequences with multiple input dimensions, which improves the prediction accuracy under complicated driving patterns and enhances the generalization performance and robustness of the model.