Self-Attention ConvLSTM for Spatiotemporal Forecasting of Short-Term Online Car-Hailing Demand

Abstract

As a flourishing basic transportation service in recent years, online car-hailing has made great achievements in metropolitan cities. Accurate spatiotemporal forecasting plays a significant role in the deployment of a network for online car-hailing demand services. A self-attention mechanism in convolutional long short-term memory (ConvLSTM) is proposed to accurately predict the online car-hailing demand. It can more effectively address the disadvantage that ConvLSTM is not good at capturing spatial correlation over a large spatial extent. Furthermore, it can generate features by aggregating pair-wise similarity scores of features at all positions of input and memory, and thus obtain the function of long-range spatiotemporal dependencies. First, the online car-hailing trajectories dataset was converted into images after geographic grid matching, and image enhancement was performed by cropping. Then, the effectiveness of the ConvLSTM embedded with a self-attention mechanism (SA-ConvLSTM) was demonstrated by comparing it to existing models. The experimental results showed that the proposed model performed better than the existing models, and including spatiotemporal information in images would perform better predictions than including spatial information in time-series pixels.

1. Introduction

Building an intelligent, green, efficient, and safe integrated transportation system requires a new form of transportation service that injects new vitality into intelligent transportation technology innovation and market expansion. The full use of app-based ridesharing mobility resources can relieve various traffic pressures and help create a new pattern of sustainable transportation. Online car-hailing is a supplement to urban public transit, and accurate prediction of it can guide users to avoid peaks and congestion, and improve the quality of passenger travel experience. The research on the spatiotemporal data of online car-hailing travel demand can not only help residents make more reasonable travel decisions and improve their travel efficiency, but also assist the city planners in the improvement of route usage and help drivers make wise travelling decisions [1].

The short-term demand forecast of online car-hailing is a typical short-term forecasting problem of time series. The short-term prediction theoretical methods and models in the traffic field in the existing research focus on the prediction of traffic flow, average speed of road sections, and so on [2]. They are mainly divided into the following four categories: prediction models based on linear system theory [3,4,5,6,7,8,9], prediction models based on nonlinear system theory [10,11,12,13], intelligent prediction models based on machine learning [14,15,16,17,18], and combined forecast models [19].

Online car-hailing is a relatively new industry, but taxis and online car-hailing have strong similarities, and the demand forecasting of both can learn from each other. Sitorus [20] et al. used a support vector machine (SVM) algorithm to analyze the obtained telematic data and predict the risk of online transportation trips. Stefano et al. [21] considered multi-source datasets, considered the correlation between travel demand and flow, and extended the Kalman Filter theory to the Local Ensemble Transformed Kalman Filter (LETKF) to predict the travel demand of online car-hailing. Nie [22] used a long short-term memory (LSTM) network to predict the demand based on regional division, which verified that the prediction accuracy was high in the actual situation. Yu et al. [23] used an LSTM model to predict the trend of car-hailing over time, taking into account the characteristics of time period, weeks, weather conditions, and so on; they found that this model can better predict the short-term demand. Jiang et al. [24] used an SVM neural network to build a least squares support vector machine (LS-SVM) based on an online car-hailing travel demand prediction model, and through experiments they showed that the prediction results of the proposed prediction model method were better than those of other methods. Rahman et al. [25] used a one-dimensional convolutional neural network (1D CNN) and zone-distributed independently recurrent neural network (IndRNN) to build a spatiotemporal deep learning structure, and tested it on real online car-hailing data; their network outperformed traditional time series models and machine learning models. In response to the analysis of the spatiotemporal correlation of the car-hailing data, Ke et al. [26] combined the ConvLSTM model and the LSTM network to extract the spatiotemporal and temporal characteristics of the travel demand of the car-hailing, and finally verified its effectiveness through experiments. Chen et al. [27] proposed a model for predicting the departure and arrival flows of taxis based on graph convolutional networks (GCN), taking road sections as nodes and taxi demand as the characteristics of nodes. Then, the departure flow prediction task and the arrival flow prediction task were combined, and the multi-task learning strategy was adopted to speed up the learning process, and a more generalized result was obtained. Geng et al. [28] proposed the spatiotemporal multi-graph convolution network (ST-MGCN) for ride-hailing demand forecasting, and the proposed model is evaluated on two real large-scale ride-hailing demand datasets and achieves over 10% improvement over state-of-the-art baselines.

The above-mentioned existing researched forecasting models have been successful in short-term forecasting of online car-hailing demand. However, there are still two main problems: (1) in the prediction of converting travel demand into images, the granularity of some models was too fine, which brought problems such as data sparseness and increased computational complexity; and (2) with the development of deep learning, more advanced models, methods, and mechanisms have been proposed, such as bidirectional LSTM (Bi-LSTM), ConvLSTM, attention mechanism, self-attention mechanism, and graph neural network (GNN). It is necessary to use new models and mechanisms to propose more reasonable prediction methods to cover different situations.

Taking the defects in previous studies as our point of departure, we add our own innovations. In summary, our contributions are summarized as follows: (1) We obtained the global spatial dependence of online car-hailing demand by adding a self-attention module in the single layer rather than superimposing a convolution operation. (2) Additional memory units in the self-attention module were used to capture the spatiotemporal information contained in the online car-hailing demand images. (3) We conducted extensive experiments on a large-scale real-world online car-hailing dataset from Didi Chuxing. The results show that our method consistently outperforms the competing baselines.

2. Related Work

The spatiotemporal prediction of online car-hailing demand based on the deep neural network models was the focus of our research. Liu et al. [29] proposed a spatiotemporal data integration model to predict online car-hailing demand, treating the predicted results as individual channels of the image; the proposed integration module first compresses the results and then restores the results using a fully convolutional network (CNN). The framework of this model can be used to improve the predictive capabilities of other models. Wang et al. [30] constructed a deep learning travel demand forecasting framework based on deep spatiotemporal ConvLSTM and CNN, and compared the three components of closeness, period, and trend to prove the validity of the model. Tan et al. [31] and Lu et al. [32] both used the ConvLSTM model for spatiotemporal prediction of online car-hailing. Yao et al. [33] proposed a deep multi-view spatiotemporal network (DMVST-Net) to model spatiotemporal relationships; the model includes three views: temporal view (through LSTM to model the correlation between future demand values and recent time points), spatial view (through local CNN to model local spatial correlations), and semantic views (to model correlations between regions with similar temporal patterns).

In previous studies on spatiotemporal prediction by converting online car-hailing demand into time series image groups, most of them chose to use the fully convolutional network model [29,33] or the ConvLSTM model [30,31,32]. However, Liu et al. [29] only provided the evaluation index of CNN prediction but did not provide the visualization result of prediction; Wang et al. [30] used heat maps to compare the predicted and true values of ConvLSTM and CNN. Although the predicted results were better in areas with large data coverage, there were deviations in low-density areas; Tan et al. [31] proposed a model with good predictive effect, but the spatial granularity of the grid is too coarse to reflect the spatiotemporal characteristics of demand in small areas; the spatial granularity of the grid set by Lu et al. [32] is too fine, and the ConvLSTM cannot capture the long-range spatiotemporal relationship, resulting in a mediocre final prediction effect. Yao et al. [33] considered the views comprehensively, but the characteristics of each perspective flattened the output forecast results through the full connection layer, showing only the difference in the temporal dimension prediction effect. In order to solve the above problems in the existing research, this paper selected the appropriate granularity according to the area size, data density, and precision during grid division; an additional memory unit was added to the original self-attention mechanism, and they were added to ConvLSTM to capture long-range spatiotemporal dependencies, and finally the spatial prediction images arranged in time series were obtained.

3. Preliminaries

In this section, we will give explicit definitions of several key concepts and formulate the modules of the model for car-hailing demand forecasting problem. The methodology framework of this study is shown in Figure 1.

3.1. Detailed Definitions

The time data have only one-dimensional attributes and are strictly sequential, so the time range can be divided into several equal-sized time windows according to the start time and interval. The spatial data generally have multi-dimensional attributes, and the spatial relationships between units are more complex, which often requires grid division.

Definition 1 (Geographic grid).

As shown in Figure 2, grid division refers to dividing areas into checkerboard-shaped grids, and classifying spatial data points into individual grids [34]. In the experiment, we divided the longitude and latitude into $p\times q$ geographic grids. In addition, the grids will correspond to the pixels in the image, and the neural networks can be used to capture the association between spatial demand information.

Definition 2 (Travel demand).

Travel demand is defined as follows [30]:

$$T{D}_t^{p,q}={\displaystyle \sum {}_{D_t\in O}}\left|\left\{O\left(p,q\right)\right\}\right|$$

(1)

where$\left|\left\{O\left(p,q\right)\right\}\right|$denotes the set of geographic grids$\left(p,q\right)$;$D_t\in O$represents the demand in the time slot$t$is belong to$O$;$T{D}_t^{p,q}$means the set of cumulative demand in the time slot$t$in$O$.

Mathematically, if travel demand is described in the form of a spatiotemporal demand matrix, then $T{D}_t^{p,q}$ can be rewritten as follows:

$$T{D}_t^{p,q}=\left[\begin{array}{cccc}{d}_t^{1,\hspace{0.33em}1}& d_t^{2,\hspace{0.33em}1}& \cdots & d_t^{p,\hspace{0.33em}1}\\ d_t^{1,\hspace{0.33em}2}& d_t^{2,\hspace{0.33em}2}& \cdots & d_t^{p,2}\\ ⋮& ⋮& \ddots & ⋮\\ d_t^{1,\hspace{0.33em}q}& d_t^{2,\hspace{0.33em}q}& \cdots & d_t^{p,q}\end{array}\right]$$

(2)

where $d_t^{p,\hspace{0.33em}q}$ denotes the cumulative demand the $\left(p,q\right)$-th grid in the time slot $t$.

Definition 3 (Demand prediction task).

According to the above, the travel demand matrix is arranged in a time-series to obtain the travel demand time-series $X_t^O$containing spatial information:

$$X_t^O=\left\{T{D}_{t-n_1\tau }^{p,q},T{D}_{t-\left(n_1-1\right)\tau }^{p,q},\cdots ,T{D}_{t-\left(n_1-n_2\right)\tau }^{p,q},\cdots ,T{D}_{t-\tau }^{p,q},T{D}_t^{p,q}\right\}$$

(3)

where $n_1$,$n_2$denotes two different time slots; $\tau$is the time interval of each time slot.

The prediction task is to forecast $\left\{T{D}_{t-\left(n_1-n_2\right)\tau }^{p,q},\cdots ,T{D}_{t-\tau }^{p,q},T{D}_t^{p,q}\right\}$ with the application of the historical demand data $\left\{T{D}_{t-n_1\tau }^{p,q},T{D}_{t-\left(n_1-1\right)\tau }^{p,q},\cdots ,T{D}_{t-\left(n_1-n_2+1\right)\tau }^{p,q}\right\}$.

3.2. ConvLSTM Model

The ConvLSTM model was born out of a precipitation prediction, as well as the problem of predicting the future rainfall intensity [35]. The researchers extended the idea of LSTM to make the state transformation in the model structure have a convolution structure. As shown in Figure 3, it consists of one memory unit and three gates, and the memory unit can be regarded as a memory update gate. The input, forget, and output gates, as well as the memory information, can be expressed by the following formulas [35]:

$${\tilde{i}}_t={\tilde{\sigma }}^i\left({\tilde{W}}_i\otimes \left[{\tilde{h}}_{t-1},{\tilde{X}}_t\right]+{\tilde{b}}_i\right)$$

(4)

$${\tilde{f}}_t={\tilde{\sigma }}^f\left({\tilde{W}}_f\otimes \left[{\tilde{h}}_{t-1},{\tilde{X}}_t\right]+{\tilde{b}}_f\right)$$

(5)

$${\tilde{o}}_t={\tilde{\sigma }}^o\left({\tilde{W}}_o\otimes \left[{\tilde{h}}_{t-1},{\tilde{X}}_t\right]+{\tilde{b}}_o\right)$$

(6)

$${\stackrel{⌢}{g}}_t=\tanh \left({\tilde{W}}_c\otimes \left[{\tilde{h}}_{t-1},{\tilde{X}}_t\right]+{\tilde{b}}_c\right)$$

(7)

$${\tilde{c}}_t={\tilde{f}}_t\odot {\tilde{c}}_{t-1}+{\tilde{i}}_t\odot {\stackrel{⌢}{g}}_t$$

(8)

$${\tilde{h}}_t={\tilde{o}}_t\odot \tanh \left({\tilde{c}}_t\right)$$

(9)

where ${\tilde{X}}_t$ denotes input in the time slot $t$; ${\tilde{i}}_t$, ${\tilde{f}}_t$, ${\tilde{o}}_t$, ${\stackrel{⌢}{g}}_t$ are the input gate, forget gate, output gate, and update gate in the time slot $t$, respectively; ${\tilde{c}}_t$ is the state of candidate memory in the time slot $t$; $\otimes$ is the process of convolution; $\odot$ is the Hadamard product; ${\tilde{\sigma }}^i$, ${\tilde{\sigma }}^f$, ${\tilde{\sigma }}^o$ are the activation function of the input gate, forget gate, and output gate, respectively; ${\tilde{W}}_i$, ${\tilde{W}}_f$, ${\tilde{W}}_c$, ${\tilde{W}}_o$ and ${\tilde{b}}_i$, ${\tilde{b}}_f$, ${\tilde{b}}_c$, ${\tilde{b}}_o$ are the weight and bias for calculating ${\tilde{i}}_t$, ${\tilde{f}}_t$, ${\stackrel{⌢}{g}}_t$, ${\tilde{o}}_t$, respectively.

3.3. Self-Attention Module

Figure 4 shows the pipeline of the self-attention module. The self-attention module receives two inputs: the input feature ${\mathcal{H}}_t$ is the hidden state in ConvLSTM in the time slot $t$, and the input feature ${\mathcal{M}}_{t-1}$ is the memory in the time slot $t-1$. The original feature maps ${\mathcal{H}}_t$ are mapped into different feature spaces as the query: $Q_h=W_{hq}{\mathcal{H}}_t\in {ℝ}^{\widehat{C}\times N}$; the key: $K_h=W_{hk}{\mathcal{H}}_t\in {ℝ}^{\widehat{C}\times N}$; and the value: $V_h=W_{hv}{\mathcal{H}}_t\in {ℝ}^{C\times N}$. The memory is mapped into key: $K_m=W_{mk}{\mathcal{M}}_{t-1}\in {ℝ}^{\widehat{C}\times N}$; and value: $V_m=W_{mv}{\mathcal{M}}_{t-1}\in {ℝ}^{C\times N}$, where $\left\{W_{hq},\hspace{0.33em}{W}_{hk},\hspace{0.33em}{W}_{hv},\hspace{0.33em}{W}_{mk},\hspace{0.33em}{W}_{mv}\right\}$ is a set of weights for $1\times 1$ convolutions, $C$ and $\widehat{C}$ are number of channels, and where $N=H\times W$, the output is ${\widehat{\mathcal{H}}}_t$ and the updated memory is ${\mathcal{M}}_t$ [36].

The whole pipeline can be separated into three parts: the feature aggregation to obtain the global information, the memory updating, and the output [36].

(1).

Feature Aggregation. The similarity scores of each pair of points are calculated by applying the matrix production as:

$$e_{\Delta }=Q_h^T{K}_{\Delta }\in {ℝ}^{N\times N},\hspace{0.33em}\Delta \in \left\{h,\hspace{0.33em}m\right\}$$

(10)

The similarity between the $i$-th point and the $j$-th point can be indexed as:

$$e_{h;\hspace{0.33em}i,\hspace{0.33em}j}=\left({\mathcal{H}}_{t,\hspace{0.33em}i}^T{W}_{hq}^T\right)\left(W_{hk}{\mathcal{H}}_{t,\hspace{0.33em}j}\right)$$

(11)

$$e_{m;\hspace{0.33em}i,\hspace{0.33em}j}=\left({\mathcal{H}}_{t,\hspace{0.33em}i}^T{W}_{hq}^T\right)\left(W_{mk}{\mathcal{M}}_{t-1,\hspace{0.33em}j}\right)$$

(12)

where the ${\mathcal{H}}_{t,\hspace{0.33em}i}$, ${\mathcal{H}}_{t,\hspace{0.33em}j}$, ${\mathcal{M}}_{t-1,\hspace{0.33em}i}$, and ${\mathcal{M}}_{t-1,\hspace{0.33em}j}$ are feature vectors with the shape $C\times 1$. Then, all weights which are used for aggregating features are normalized by applying SoftMax function along columns:

$${\alpha }_{\Delta ;\hspace{0.33em}i,\hspace{0.33em}j}=\frac{\exp e_{\Delta ;\hspace{0.33em}i,\hspace{0.33em}j}}{{\displaystyle \sum {}_{k=1}^N\exp e_{\Delta ;\hspace{0.33em}i,\hspace{0.33em}k}}},\hspace{0.33em}\Delta \in \left\{h,\hspace{0.17em}m\right\},\hspace{0.33em}i,\hspace{0.33em}j\in \left\{1,\hspace{0.33em}2,\hspace{0.33em}\cdots ,\hspace{0.33em}N\right\}$$

(13)

The aggregated feature of the $i$-th location is calculated by a weighted sum across all locations $N$:

$$Z_{\Delta ;\hspace{0.33em}i}={\displaystyle \sum _{j=1}^N{\alpha }_{\Delta ;\hspace{0.33em}i,j}}{V}_{\Delta ;\hspace{0.33em}j}=\left\{\begin{array}{c}{\displaystyle \sum _{j=1}^N{\alpha }_{h;\hspace{0.33em}i,\hspace{0.33em}j}\left(W_{hv}{\mathcal{H}}_{t,\hspace{0.33em}j}\right)}\\ {\displaystyle \sum _{j=1}^N{\alpha }_{m;\hspace{0.33em}i,\hspace{0.33em}j}\left(W_{mv}{\mathcal{M}}_{t-1,\hspace{0.33em}j}\right)}\end{array}\right.$$

(14)

Finally, the aggregated feature $Z$ can be obtained with $Z=W_z\left[Z_h;\hspace{0.33em}{Z}_m\right]$.

(2).

Memory Updating. A gating mechanism is used to update the memory $\mathcal{M}$ adaptively, so that the self-attention module can capture long-range dependencies in terms of spatial and temporal domains. The aggregated feature $Z$ and the original input ${\mathcal{H}}_t$ are used to produce values of the input gate ${i^{\prime }}_t$ and the fused feature ${g^{\prime }}_t$. Besides, the forget gate is replaced as $1-{i^{\prime }}_t$ to reduce parameters. The updating progress can be formulated as follows [36]:

$${i^{\prime }}_t=\sigma \left(W_{m;\hspace{0.33em}zi}\otimes Z+W_{m;\hspace{0.33em}hi}\otimes {\mathcal{H}}_t+b_{m;\hspace{0.33em}i}\right)$$

(15)

$${g^{\prime }}_t=\tanh \left(W_{m;\hspace{0.33em}zg}\otimes Z+W_{m;\hspace{0.33em}hg}\otimes {\mathcal{H}}_t+b_{m;\hspace{0.33em}g}\right)$$

(16)

$${\mathcal{M}}_t=\left(1-{i^{\prime }}_t\right)\odot {\mathcal{M}}_{t-1}+{i^{\prime }}_t\odot {g^{\prime }}_t$$

(17)

Compared with the original memory cell $c$ in the ConvLSTM which is updated by convolution operations only, the memory $\mathcal{M}$ is updated by not only convolution operations but also aggregated features $Z_t$, obtaining the global spatial dependency in a timely manner. Therefore, we argue that ${\mathcal{M}}_{t-1}$ is able to contain global past spatiotemporal information.

(3).

Output. Finally, the output feature ${\widehat{\mathcal{H}}}_t$ is a dot product between the output gate ${o^{\prime }}_t$ and updated memory ${\mathcal{M}}_t$, which can be formulated as follows [36]:

$${o^{\prime }}_t=\sigma \left(W_{m;\hspace{0.33em}zo}\otimes Z+W_{m;\hspace{0.33em}ho}\otimes {\mathcal{H}}_t+b_{m;\hspace{0.33em}o}\right)$$

(18)

$${\widehat{\mathcal{H}}}_t={o^{\prime }}_t\odot {\mathcal{M}}_t$$

(19)

3.4. Self-Attention ConvLSTM Model

We embed the self-attention module into the ConvLSTM to construct the SA-ConvLSTM, as illustrated in Figure 5. If we remove the SAM module, the SA-ConvLSTM will degenerate into the standard ConvLSTM. The internal calculation formulas for the self-attention module have been introduced in Section 3.3. Figure 5 is an overall introduction to the units of SA-ConvLSTM, which can be represented by the following formulas [36]:

$${\stackrel{⌢}{X}}_t=SA\left({\tilde{X}}_t\right)$$

(20)

$${\stackrel{⌢}{\mathcal{H}}}_{t-1}=SA\left({\tilde{h}}_{t-1}\right)$$

(21)

$$i_t={\sigma }^i\left(W_i\otimes \left[{\stackrel{⌢}{\mathcal{H}}}_{t-1},{\stackrel{⌢}{X}}_t\right]+b_i\right)$$

(22)

$$f_t={\sigma }^f\left(W_f\otimes \left[{\stackrel{⌢}{\mathcal{H}}}_{t-1},{\stackrel{⌢}{X}}_t\right]+b_f\right)$$

(23)

$$o_t={\sigma }^o\left(W_o\otimes \left[{\stackrel{⌢}{\mathcal{H}}}_{t-1},{\stackrel{⌢}{X}}_t\right]+b_o\right)$$

(24)

$$g_t=\tanh \left(W_c\otimes \left[{\stackrel{⌢}{\mathcal{H}}}_{t-1},{\stackrel{⌢}{X}}_t\right]+b_c\right)$$

(25)

$${\stackrel{⌢}{c}}_t=f_t\odot {\stackrel{⌢}{c}}_{t-1}+i_t\odot g_t$$

(26)

$${\stackrel{⌢}{\mathcal{H}}}_t=o_t\odot \tanh \left({\stackrel{⌢}{c}}_t\right)$$

(27)

In the experiment, a Batch Normalization layer was added after each SA-ConvLSTM model [37]. On the one hand, it can reduce the training time, conduct deep network training, and use a larger learning rate; on the other hand, it can make the learning process less affected by initialization, prevent overfitting, and reduce the need for regularization. The formulas for this process are shown as follows [37]:

For the $l$-th layer with $a$-dimensional input $x_l=\left(x_l^{\left(1\right)},x_l^{\left(2\right)},\cdots ,x_l^{\left(a\right)}\right)$, each dimension will be normalized using the following formula:

$${\stackrel{⌢}{x}}_l^{\left(b\right)}=\frac{{\stackrel{⌢}{x}}_l^{\left(b\right)}-E\left[{\stackrel{⌢}{x}}_l^{\left(b\right)}\right]}{\sqrt{Var\left[{\stackrel{⌢}{x}}_l^{\left(b\right)}\right]}}$$

(28)

where ${\stackrel{⌢}{x}}_{{}_l}^{\left(b\right)}$ denotes each activation; $E\left[\cdot \right]$ stands for the calculated mean; $Var\left[\cdot \right]$ represents the calculated variance.

For each activation ${\stackrel{⌢}{x}}_{{}_l}^{\left(b\right)}$, a pair of parameters will be used to scale and shift the normalized value:

$$y_l^{\left(b\right)}={\gamma }_l^{\left(b\right)}{\stackrel{⌢}{x}}_l^{\left(b\right)}+{\beta }_l^{\left(b\right)}$$

(29)

where ${\gamma }_{{}_l}^{\left(b\right)}$ and ${\beta }_{{}_l}^{\left(b\right)}$ denotes the parameters to be learned along with the original model parameters, and they restore the representation power of the network.

In the experiment, we used the three-dimensional convolution (Conv3D) to output the predicted image group. The specific experimental process is shown in Figure 6, where the image enhancement method will be introduced in the next section.

4. Experiments

In this section, we will introduce methods on how to process the online car-hailing trajectories data and fully extract the required spatiotemporal information from them, and then we will verify the effectiveness of the proposed model and compare it with existing methods.

4.1. Dataset

The dataset used in the experiments contained real-world online car-hailing trajectories from the city of Chengdu, a large city in southwestern China [38]. It was provided by Didi Chuxing GAIA Initiative, with online car-hailing trajectories data from 1 November 2016 to 30 November 2016. An example of the original online car-hailing trajectories data is shown in

Table 1. Example of original online car-hailing trajectories data.

Driver ID	Order ID	Timestamp	Longitude	Latitude
XXXXXX	XXXXXX	1477969147	104.0751	30.72724

, where the first column and the second column both show encryption processing.

After the transformation of the spatial coordinate system, the spatial and temporal features hidden in the data are mined and the spatiotemporal data are fused by dividing the spatial and temporal units. The spatiotemporal grid division process of a trajectory is shown in the Figure 7. For the online car-hailing trajectories data, while classifying the positioning points, the vector trajectories are also rasterized into grid sequences, a method that realizes the process of scattering the trajectories data into point coordinates and converting them into tensors. Each grid unit is a pixel in the image. The smaller the granularity of the pixel, the higher the definition. When the granularity of the grid is fine to a certain level, it can truly and accurately reflect the features in the area. However, too small granularity can cause problems such as data sparseness and increased computational complexity.

First, the longitude of 104.081083° to 104.111122° and the latitude of 30.653965° to 30.674219° were selected for research, each geographic grid cell was 2500 m² in size, and the research area was divided into 53 × 67 grids. Figure 8 illustrates the process of trajectory data acquisition and image conversion. Second, with a time interval of 10 min, the online car-hailing demand in different regions was counted. Three cumulative demand images are shown in Figure 8, which are the time units 00:00, 11:00, and 23:00 on 1 November. These three images from the spatial dimension yield the observation that there are many grids with missing data. There are three main reasons for this incomplete data: first, the travel proportion of online car-hailing is not high, and the vehicle trajectories cannot cover all areas; second, a large amount of land in the city is non-urban roads such as vegetation or buildings, and the demand of online car-hailing is 0; and third, the grid size is too small. Similarly, in the time dimension, there are also a lot of vacancies in data, which is caused by fluctuations of the demand for online car-hailing in time. For example, at midnight, the demand for online car-hailing is far less than during the morning and evening rush hours.

As mentioned above, due to the fine granularity of the initial grid, many grids lack data, or the volume in the grid is extremely low. Data with values that are too small are susceptible to random perturbations and are unpredictable. In addition, for the online car-hailing service platform, the fine-grained spatiotemporal grid is not necessarily conducive to the scheduling of vehicles, which may bring great scheduling difficulties to the scheduling system. Therefore, the initial spatiotemporal grids need to be aggregated. In the spatial dimension, 2 × 2 original grids were merged into a new grid; in the time dimension, the original 6 time periods were merged into a new grid time granularity. We completed missing grids and filled in “0” for missing values that existed in grid granularity. The above spatiotemporal integration operations are shown in Figure 9. In addition, the online car-hailing dataset covers only one month. In order to expand the number of samples and improve the training effect, it is necessary to perform image enhancement on the training data. One of the most commonly used image enhancement methods is cropping [39]. As shown in the Figure 10, the experiment uses a square sliding window with a length and width of 20 to further clip the spatial grid, increasing the amount of data to 120 times of the original.

4.2. Evaluation Metrics

We use four valuation metrics that are commonly used to evaluate the traffic forecasting performance: Structural Similarity (SSIM), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Coefficient of Determination (R²), which can be defined as follows:

$$\mathrm{SSIM}={\left[S_1\left(T{D}_t^i,{\widehat{TD}}_t^i\right)\right]}^{\omega }{\left[S_2\left(T{D}_t^i,{\widehat{TD}}_t^i\right)\right]}^{\xi }{\left[S_3\left(T{D}_t^i,{\widehat{TD}}_t^i\right)\right]}^{\psi }$$

(30)

$$\mathrm{MAE}=\frac{1}{U}{\displaystyle \sum _{i=1}^U\left|T{D}_t^i-{\widehat{TD}}_t^i\right|}$$

(31)

$$\mathrm{RMSE}=\sqrt{\frac{1}{U}{\displaystyle \sum _{i=1}^U{\left(T{D}_t^i-{\widehat{TD}}_t^i\right)}^2}}$$

(32)

$${\mathrm{R}}^2=1-\frac{{\displaystyle {\sum }_{i=1}^U{\left(T{D}_t^i-{\widehat{TD}}_t^i\right)}^2}}{{\displaystyle {\sum }_{i=1}^U{\left(T{D}_t^i-{\overline{TD}}_t^i\right)}^2}}$$

(33)

where $T{D}_t^i$ and ${\widehat{TD}}_t^i$ represent the true value and predicted value of the $i$-th grid in the time slot $t$; ${\overline{TD}}_t^i$ represents the mean value of the sample; $U$ is the total number of geographic grids; $S_1\left(T{D}_t^i,{\widehat{TD}}_t^i\right)$, $S_2\left(T{D}_t^i,{\widehat{TD}}_t^i\right)$, $S_3\left(T{D}_t^i,{\widehat{TD}}_t^i\right)$ represent the pixel mean, variance, and covariance of the true and predicted images, respectively, and they also represent comparative brightness, contrast, and structure, respectively; $\omega$, $\xi$, $\psi$ are parameters to adjust the relative importance of $S_1\left(T{D}_t^i,{\widehat{TD}}_t^i\right)$, $S_2\left(T{D}_t^i,{\widehat{TD}}_t^i\right)$, $S_3\left(T{D}_t^i,{\widehat{TD}}_t^i\right)$ respectively.

4.3. Training Configuration

In this section, we introduce some parameter settings and details on the proposed models. Each model is trained with an Adam algorithm and the learning rate of it is $0.001$. During the training process, the batch size is set to $32$; the number of training epochs is set to $50$; the activation function in each layer of Conv3D adopts the Tanh function; and the activation function in each layer of ConvLSTM and SA-ConvLSTM adopts the SoftMax function; the activation function of each layer of the comparison model with time-series pixels as input adopts the ReLU function. In the experiment, each layer of ConvLSTM and SA-ConvLSTM had $10$ filters with the size of each filter being $3\times 3$; when the Conv3D layers were used as the input and hidden layers, there were $10$ filters per layer; and when the Conv3D layer was used as the output layer, there was only one filter per layer, each of which had a size of $3\times 3\times 3$. An early stop strategy was used to prevent model overfitting and end the training. The loss functions used in the models to update the parameters in the models were Mean Square Error (MSE). In the experiment, TensorFlow and Keras were used to implement the deep learning models described in Section 3.2 and Section 3.4.

Section 3.1 and Section 4.1 were combined and resulting size of the image was $20\times 20\times 1$. Each frame contains a total of $24$ images as input and output groups, of which the first $12$ images were used as input, and the last $12$ were used as output. The experiments set $2880$ images from the last day of $30$ days as the test set, and the images from the remaining $29$ days were divided into training and validation sets in a ratio of 8 to 2. The model structure of SA-ConvLSTM is shown in

Table 2. The model structure of SA-ConvLSTM.

Layer (Type)	Output Shape	Parameter
InputLayer	(None, None, 20, 20, 1)	0
SaConvLSTM2D_1	(None, None, 20, 20, 10)	4800
BatchNormalization_1	(None, None, 20, 20, 10)	40
SaConvLSTM2D_2	(None, None, 20, 20, 10)	8040
BatchNormalization_2	(None, None, 20, 20, 10)	40
SaConvLSTM2D_3	(None, None, 20, 20, 10)	8040
BatchNormalization_3	(None, None, 20, 20, 10)	40
Conv3D	(None, None, 20, 20, 1)	271

.

4.4. Analysis and Discussion of Prediction Performance

In this section, in order to prove the validity of the proposed model, we compared it with previous model algorithms. Among them, for the image prediction containing spatiotemporal information, the ConvLSTM and CNN models with good effect in spatiotemporal prediction were selected; for the time-series pixel prediction containing spatial information, the LSTM, GRU, Bi-LSTM, and bidirectional GRU (Bi-GRU) models with good performance in time-series prediction were selected. With regard to the specific method of time-series pixel prediction, we, first, flattened the pixels in each image, and then the vectors were arranged according to the grid location of each pixel; second, we arranged each vector in chronological order to form time-series vectors; finally, according to the settings in the experiment, every 12 historical time-series vectors were used to predict the next 12 time-series vectors.

As shown in

Table 3. Performance comparison of different models for demand prediction.

Model	Performance Indices
	SSIM (%)	MAE	RMSE	R2 (%)
LSTM	32.97	75.47	134.03	40.37
GRU	34.25	73.85	133.36	41.89
Bi-GRU	46.22	52.92	101.26	65.56
Bi-LSTM	48.03	52.81	101.34	65.62
CNN	93.00	27.11	51.78	91.23
ConvLSTM	94.47	20.86	43.31	93.83
SA-ConvLSTM	95.15	19.31	38.62	95.11

, the proposed SA-ConvLSTM prediction network obtained the lowest MAE and RMSE, as well as the highest SSIM and R², mainly because our method accounts for both the local and global correlations between the online car-hailing demand in the research area. It fully explored the spatial and temporal characteristics. Although LSTM and GRU could extract the long-term dependence effectively, it was not good at spatial feature mining. Compared to LSTM and GRU, the bi-directional mechanism made Bi-LSTM and Bi-GRU more superior in extracting the temporal features of the periodicity of time-series pixels and the contextual relationship of pixels, but they were also insufficient in capturing spatial features. The CNN was not good at temporal feature mining. ConvLSTM was superior to them in spatiotemporal feature mining. Figure 11 shows the prediction results of the 1st, 40th, 80th, and 120th cropped image sub-images at 12:00 on 30 November, respectively. The self-attention module that considers additional memory units was effective.

In the comparative experiment, we compared the spatiotemporal prediction models of online car-hailing demand images proposed in previous studies in Section 2 to the proposed model, and obtained the following conclusions: the image enhancement method not only increased the sample size, but also solved the problems of fine spatial granularity and irregular image size; SA-ConvLSTM can capture long-range spatial dependencies and achieve good prediction results. Therefore, our research is not only applicable to the prediction of online car-hailing demand, but also can be implemented for object detection in traffic scenes, such as vehicle detection, traffic light detection, and pedestrian detection. The images obtained from traffic scenes are complex, good prediction results are not only useful for automatic driving, but also help drivers drive safely. However, the parameters of SA-ConvLSTM are larger than those of ConvLSTM and CNN, and the training time is longer, which makes our equipment unable to load experiments with larger images.

5. Conclusions

With the purpose of predicting short-term online car-hailing demand, this study investigated a variant model of ConvLSTM based on a self-attention mechanism, which mainly solves the complex spatiotemporal prediction problem. The model used additional memory units on the traditional self-attention mechanism to enable the model to capture long-term time dependence. Experiments showed that adding a self-attention module is more efficient than stacking convolutional layers. Compared to convolution operations, the self-attention module showed the capacity to capture the features of special locations far away from the data concentration area, as well as the ability to remember the global temporal and spatial dependence.

The reasonable forecast of demand distribution also improves the travel efficiency of users, and more resources have been effectively utilized without increasing supply. With the development of machine learning, new mechanisms and models continue to emerge, such as transformer models, which can be considered in future work. In addition, more kinds of data related to ride-hailing demand should be considered, such as season, customer income, charging policy, point of interest data, etc. In the actual road network, the complex and large-scale online car-hailing demand will be difficult to predict, and more suitable deep learning models are needed to solve it, such as a class based on graph convolutional networks [27]. Moreover, the complex trajectory data contains many characteristics of traffic conditions and driver behavior; for example, some features of route selection decision can be extracted from it [40], or the behavior of traffic participants can be estimated and their future trajectories can be predicted [41,42]. Therefore, these will be considered in our future work.