A Game-Theory-Based Approach to Modeling Lane-Changing Interactions on Highway On-Ramps: Considering the Bounded Rationality of Drivers

Abstract

In highway on-ramp sections, the conflictual interactions between a subject vehicle (merging vehicle) in the acceleration lane and a following vehicle (lagging vehicle) in the adjacent mainline can lead to traffic congestion, go–stop oscillations, and serious safety hazards. Human drivers combine their previous lane-changing experience and their perception of surrounding traffic conditions to decide whether to merge. However, the decisions that they make are not always optimal in specific traffic scenarios due to fuzzy perception and misjudgment. That is, they make lane-changing decisions in a bounded rational way. In this paper, a game-theory-based approach is used to model the interactive behavior of mandatory lane-changing in a highway on-ramp section. The model comprehensively considers vehicle interactions and the bounded rationality of drivers by modeling lane-changing behavior on on-ramps as a two-person non-zero-sum non-cooperative game with incomplete information. In addition, the Logit QRE is used to explain the bounded rationality of drivers. In order to estimate the parameters, a bi-level programming framework is built. Vehicle trajectory data from NGSIM and an unmanned aerial vehicle survey were used for model calibration and validation. The validation results were rigorously evaluated by using various performance indicators, such as the mean absolute error, root mean square error, detection rate, and false-alarm rate. It can be seen that the proposed game theory-based model was able to effectively predict merging and yielding interactions with a high degree of accuracy.

1. Introduction

Car-following behavior in single-lane traffic and lane-changing behavior in multi-lane traffic are two important topics for traffic flow modeling. Lane-changing behavior involves single-vehicle transverse and longitudinal movement and multi-vehicle interactions, which are crucial to the traffic flow. Lane-changing drivers need to interact with other drivers and the surrounding environment to make changing decisions. Conflicting decisions can lead to traffic congestion, go–stop oscillations, and serious safety hazards [1]. On the other hand, a cooperative decision helps maintain the stability of the traffic system.

Previous studies classified lane-changing behavior into two categories: discretionary lane-changing for pursuing a better driving environment and mandatory lane-changing in order to move into the planned lane. In the on-ramp section of a highway, the driver of a subjective vehicle (SV) in the acceleration lane needs to engage in a series of conflicting or cooperative interactions with the following vehicle (FV) on the mainline to merge smoothly into the mainline, which can be defined as a typical mandatory lane-changing scenario. This is likely to cause deceleration in the FV and has a significant impact on the mainline traffic flow. This study focuses on modeling and analyzing this behavior to reduce the disruptions caused by on-ramp merging.

Existing studies on mandatory lane-changing on on-ramps mainly focused on lane-changing intention recognition [2,3,4], lane-changing trajectory prediction [5,6], and lane-changing controller design [7,8,9]. However, the study of drivers’ lane-changing decision characteristics, especially the modeling of drivers’ bounded rational consideration, has not been sufficiently addressed. In fact, drivers do not always have the correct expectations or beliefs about other drivers’ decisions, and their judgments about the surrounding environment and their own lane-changing experience are important factors affecting lane-changing decisions. However, the strategy that is chosen is not always the optimal solution in the current situation, that is, human drivers make decisions under bounded rationality.

The objective of this paper is to propose a mandatory lane-changing decision-making mechanism that takes the bounded rationality of human drivers into account. We extend Arb’s work by using a Quantum Reaction Equilibrium (QRE) solution to characterize bounded rationality [10]. The QRE model takes a driver’s judgment errors and other unobservable errors into account so as to optimize the reflection of merging interactions. That is, the player (human driver) is assumed to have correct average beliefs and tends to choose the strategy with the highest utility, but still makes mistakes according to the probability distribution.

We define the game structure of a merging interaction as a two-person, simultaneous, non-cooperative, and non-zero-sum game. The merging and waiting strategies of the SV and the acceleration, deceleration, and do-nothing strategies of the FV are taken into account. The proposed model is calibrated and validated with a vehicle trajectory dataset collected along a section of Interstate 80 in Emeryville, California under the NGSIM program [11] and a vehicle trajectory dataset collected from an unmanned aerial vehicle (UAV) survey conducted on Xi’an Ring Highway, Xi’an, Shaanxi, China. Finally, the prediction results of the model are given, and the conclusions and suggestions for further research are summarized.

The remainder of this paper is organized as follows: Section 2 reviews the existing lane-changing studies. Section 3 introduces the definition of the on-ramp mandatory lane-changing game and gives the payoff formulation for the SV and FV. A description of the data and processing methods is presented in Section 4. Section 5 presents the model calibration and validation. Finally, the summary and future possible research directions are discussed in Section 6.

2. Literature Review

In previous studies, the lane-changing modeling methodologies can be divided into rule-based models, utility-theory-based models, artificial-intelligence-based models, and game-theory-based models. For the sake of this study, we first briefly introduce the rule-based and utility-based lane-changing models, and then give a specific review of game-theory-based models.

2.1. Rule-Based Models

Rule-based models first evaluate the possible reasons for lane-changing, and then use a gap acceptance model to determine whether the gaps should be accepted [12]. Gipps [13] first proposed a rule-based decision tree framework for lane-changing behavior. Multiple impact factors, such as changing safety, vehicle type, and positions of barriers, were contained in his framework. Based on Gipp’s work, some scholars further divided the lane-changing process (i.e., lane-changing consideration, target lane determination, acceptable gap searching, and lane-changing operation) [14] and lane-changing types (i.e., mandatory lane-changing, discretionary lane-changing, and cooperative lane-changing) [15]. Some researchers also proposed cooperative merging strategies [16] and algorithms [17] in a connected environment.

2.2. Utility-Theory-Based Models

A utility-based model evaluates the feasibility of lane-changing behavior through the utility values of drivers for various benefits and risks. Ahmed [18] considered driver heterogeneity, proficiency, and decision-making state and established a utility function to estimate the benefits of lane-changing and continuing to follow. Toledo et al. [19] combined mandatory lane-changing and discretionary lane-changing into one benefit function and introduced unobserved variables to reflect driver heterogeneity.

2.3. Game-Theory-Based Models

Compared with rule-based or utility-based models, a game-theory-based model incorporates the decisions of the subject and the immediate follower in the mainline in a competing situation, where the decision of one decision maker depends on the actions of the other. This can better describe the lane-changing process based on the interaction between drivers based on a payoff function, this has a stronger interpretability and universality than those of artificial-intelligence-based models.

Kita [20] first modeled a vehicle interaction as a two-player non-zero-sum non-cooperative game. Each player had two strategies (merge or wait for the SV, give way or not for the FV) in his game, and the player payoff was formulated by the safety criterion of time to collision. On this basis, other researchers made further efforts by taking the safety requirements [1,21,22,23], comfort requirements [9], and vehicle types [10] into consideration, thus expanding the strategy set and payoff formulation for the on-ramp merging game. In a recent study, Ali et al. [24] proposed a complete lane-changing modeling framework that could explain both mandatory and discretionary lane-changing behaviors. Shao et al. [25] considered the heterogeneity and endogeneity of human drivers by using a signaling-game-based approach to modeling the lane-changing decision-making mechanism. With the development of communication and information technology, drivers or auto-control systems can better perceive the surrounding traffic conditions and even communicate with each other. Thus, merging games in a connected environment [1,9,24] and automated environment [7,26] are increasingly being developed. The partial uncertainty in mixed traffic modeling is also a focal point [27,28,29].

Based on the absolute rational person hypothesis, researchers constructed various specific game models to explain drivers’ decision-making characteristics. The Nash equilibrium is achieved when no player can unilaterally increase their expected payoff by changing their strategy. This can be divided into pure strategy games and mixed strategy games. In a pure strategy game [1,23,24], drivers select a certain strategy for the best payoff. A mixed strategy game [10,30,31] assigns a probability for each driver’s strategy, and the driver chooses to maximize their own payoff according to the probabilities set. Yu et al. [7] adopted the Stackelberg game structure, assuming that the SV on on-ramps has the game advantage. That is, the FV makes its decision after the SV does. They developed a gap selection model and validated the interaction in a simulator. Kang and A. Rakha [31] used a repeated game framework to update the payoff function. The validation results when using NGSIM data showed that the model provided better prediction accuracy.

In a traditional environment, it is hard for human drivers to accurately perceive the surrounding environment. Drivers often make decisions in a bounded rational way, and the surrounding traffic environment and past lane-changing experience will affect the accuracy of their decisions. Only a few studies took this feature into consideration [10]. Meanwhile, with the development of mandatory lane-changing game studies, more strategies, such as acceleration, deceleration, doing nothing, and lane-changing, have been modeled [1]. In order to model bounded rationality and enrich players’ strategy set, the QRE game is extended. We will present the methodologies and experiments in the following sections.

3. Methodology

3.1. Game Definition

This study describes a mandatory lane-changing decision-making process in a highway on-ramp section in a traditional environment. Consider a typical lane-changing situation in the highway on-ramp section shown in Figure 1, where the players involved include a merging vehicle, that is, the subject vehicle (SV) on the acceleration lane, an immediately following/lagging vehicle (FV), and a possible leading vehicle on the target mainline. In the traditional manual driving environment, the SV driver looks at the rear-view mirror and side-view mirrors to check if the surrounding traffic is suitable for merging, and then uses a turn signal or makes a small lateral move to send a lane-changing signal. Meanwhile, the FV driver responds to the signal by accelerating, decelerating/yielding, or doing nothing. However, some aggressive SV drivers may still make the merging operation despite the FV’s acceleration, and FV drivers may keep their car-following state with respect to the leading vehicle or accelerate to block the merging gap though the SV has already started the merging maneuver. Actually, both the SV and FV make decisions based on surrounding traffic conditions and their own experience, which may not result in the optimal solution in the specific scenario and may cause a traffic conflict. Thus, they are both assumed to perform in a bounded rational way.

In this study, the interaction between the SV and FV is regarded as a simultaneous, two-person, non-cooperative, non-zero-sum game with incomplete information. A simultaneous game means that the on-ramp and mainline players make their decisions at the same time. In the non-cooperative and non-zero-sum situation, all players are in a competitive relationship, and the sum of their payoffs corresponding to the actions is not zero. Human drivers rely on vision and hearing to make inaccurate judgments about others’ states, so this is a situation with incomplete information. This game-theoretical structure has also been adopted in existing study [1].

To get a better description of the interaction of the SV and FV, this study enriches the strategy set of the FV based on [10] and constructs six typical scenarios, which are shown in

Table 1. Typical scenarios of the on-ramp merging game in its normal form.

	Player 2: SV
	Actions	Merge $\left({\mathit{S}}_1\right)$	Wait $\left({\mathit{S}}_2\right)$
Player 1: FV	Accelerate/Block $\left(F_1\right)$	$(P_{11},Q_{11})$	$(P_{12},Q_{12})$
	Do Nothing $\left(F_2\right)$	$(P_{21},Q_{21})$	$(P_{22},Q_{22})$
	Decelerate/Yield $\left(F_3\right)$	$(P_{31},Q_{31})$	$(P_{32},Q_{32})$

. The SV driver can choose to merge immediately or wait for the next available gap, and the FV driver’s strategies can be divided into “accelerating” to block the merging gap, “doing nothing” to keep their following status, or “decelerating” to courteously yield. Though other vehicles also have an influence during the merging process, the dominant interaction is between these two vehicles [21], and the influence of the potential leading vehicle in the mainline is considered in the FV’s payoff.

It is noted that prior research made a dominant assumption concerning the game solutions of driver maneuvers in which the interactive behaviors of vehicles were presented by a mathematical Nash equilibrium. However, the hypothesis of the rational human in the Nash equilibrium suggests that a driver never selects a strategy with less return. In reality, it remains the case that, sometimes, a person may select a poor strategy due to their misjudgments. In this study, we assume that drivers make choices in a rational way, but sometimes make errors according to a probability distribution. The Quantal Response Equilibrium game solution is used to explain bounded rational behaviors caused by the misjudgments or imperfect vision of human drivers so as to smooth the sharp optimal response in standard game theory and optimize the reflection of vehicle interactions in reality.

The Quantal Response Equilibrium is a generation of the Nash equilibrium; it requires beliefs to match the equilibrium choice probability, converges to the Nash equilibrium as the quantal response functions become very steep, and approximates the best response functions [32]. The Logit Quantal Response Equilibrium (LQRE) is the most common specification for the QRE. Players’ strategies are chosen based on the probability distribution shown in Equation (1).

$$P{r}_i^{a_i}=\frac{exp\left(\lambda U_i^{a_i}\left(P{r}_{-i}\right)\right)}{{}_{\sum a_i\in A_i}exp\left(\lambda U_i^{a_i}\left(P{r}_{-i}\right)\right)}$$

(1)

where $P{r}_i^{a_i}$ is the probability of player i choosing the action $a_i$. The tuning parameter $\lambda$ can express bounded rational behaviors. When the parameter is small, the player tends to have less rational behaviors, and as $\lambda$ goes up, the player demonstrates more rational behaviors. $U_i^{a_i}$ is the expected utility of player i, and $P{r}_{-i}$ is the probability distribution of the other players.

3.2. Payoff Formulation

Since game theory was introduced to on-ramp merging scenarios, payoff functions have become an important component of research. First, payoff functions were often formulated in terms of time to collision (TTC) in order to minimize the collision risks for both sides [20]. Liu et al. [21] took the different targets of mainline and merging drivers into consideration and assumed that the lagging mainline vehicle’s object was to minimize speed variations; the merging vehicle tried to minimize the time spent in the acceleration lane under safety constraints. Payoff functions were formulated on the basis of merging safety, expected travel time, efficiency, and acceleration in Kang’s model [23]. Ali et al. [1] pointed out that the previous work may result in a trivial equilibrium solution due to the inhomogeneity of payoff units (lane-changing time for the SV and acceleration for the FV), so they used the projected acceleration to formulate the payoff.

The vehicle interaction behaviors’ payoff in this study follows the expected utility theory. The SV’s decision utilities were formulated according to the acceleration for merging into the mainline or waiting for the next available gap, and the FV’s decision utilities depended on the acceleration required to block the merging gap (i.e., acceleration), show a courtesy yield (i.e., deceleration), or keep the following state (i.e., doing nothing). This payoff formulation was also used by Ali et al. [1,24]. The following assumptions are made in a typical merging scenario: (a) Before the merging event, the FV keeps the following state with respect to the leading vehicle; (b) the SV and FV construct their payoff matrices when the SV appears in the acceleration lane; (c) the distance between the SV and FV is less than 60 m because the vehicle interaction decreases when it exceeds this range [19,21]; (d) both drivers make their decisions in a bounded rational way.

3.2.1. Payoffs for the FV

The FV driver has three strategies for responding to the SV’s action at the decision time. They can perform a courtesy yield, do nothing, or block the merging gap to prevent the SV’s lane-changing maneuver. Considering safety limitations and speed variations as the two main motives, this study assumes that in the mandatory merging case, the FV may have to brake to create a safe gap for the merging vehicle to change to the mainline. If the FV driver chooses the deceleration or do-nothing strategies, they will get the payoff for the required deceleration.

Table 2. Payoff matrices for the FV and SV.

Players		Player 2: SV
Payoff for FV	Player 1: FV	Actions	Merge ($S_1$)	Wait ($S_2$)
		Accelerate ($F_1$)	$Q_{11}={\alpha }_{11}^0+{\alpha }_{11}^{1}Ac{c}_{MA}+{\epsilon }_{11}$	$Q_{12}={\alpha }_{12}^0+{\alpha }_{12}^{1}Ac{c}_{WA}+{\epsilon }_{12}$
		Do nothing ($F_2$)	$Q_{21}={\alpha }_{21}^0+{\alpha }_{21}^{1}Ac{c}_{MDN}+{\epsilon }_{21}$	$Q_{22}={\alpha }_{22}^0+{\alpha }_{22}^{1}Ac{c}_{WDN}+{\epsilon }_{22}$
		Decelerate ($F_3$)	$Q_{31}={\alpha }_{31}^0+{\alpha }_{31}^{1}Ac{c}_{MD}+{\epsilon }_{31}$	$Q_{32}={\alpha }_{32}^0+{\alpha }_{32}^{1}Ac{c}_{WD}+{\epsilon }_{32}$
Payoff for SV	Player 1: FV	Actions	Merge ($S_1$)	Wait ($S_2$)
		Accelerate ($F_1$)	$P_{11}={\beta }_{11}^0+{\beta }_{11}^{1}Ac{c}_{MA}^{\prime }+{\delta }_{11}$	$P_{12}={\beta }_{12}^0+{\beta }_{12}^{1}Ac{c}_{WA}^{\prime }+{\delta }_{12}$
		Do nothing ($F_2$)	$P_{21}={\beta }_{21}^0+{\beta }_{21}^{1}Ac{c}_{MDN}^{\prime }+{\delta }_{21}$	$P_{22}={\beta }_{22}^0+{\beta }_{22}^{1}Ac{c}_{WDN}^{\prime }+{\delta }_{22}$
		Decelerate ($F_3$)	$P_{31}={\beta }_{31}^0+{\beta }_{31}^{1}Ac{c}_{MD}^{\prime }+{\delta }_{31}$	$P_{32}={\beta }_{32}^0+{\beta }_{32}^{1}Ac{c}_{WD}^{\prime }+{\delta }_{32}$

shows the payoff matrix for the FV and SV. In this table, $Acc$ and $Ac{c}^{\prime }$ stand for the FV’s and SV’s acceleration in different situations; $W$ and $M$ represent waiting and merging, respectively; $A,D,$ and $DN$, respectively, indicate acceleration, deceleration, and doing nothing; $\epsilon$ and $\delta$ represent the error terms that capture the unobserved variations, and they are assumed to obey a standard normal distribution, $N(0,1)$; $\alpha$ and $\beta$ are parameters to be estimated.

We formulate the specific initial states and projected states with Newtonian equations, which were also accepted by [1,21,24]. The projected states are formulated as per Equation (2).

$$\begin{array}{cc}\hfill & v_{SV}^{{}^{\prime }}=\sqrt{\left(v_{SV}^2\right)+2a_{SV}RD}\hfill \\ \hfill & t_{SV}^{\prime }=\frac{v_{SV}^{\prime }-v_{SV}}{a_{SV}}\hfill \\ \hfill & v_{FV}^{{}^{\prime }}=v_{FV}+a_{FV}{t}_{SV}^{\prime }\hfill \\ \hfill & X^{{}^{\prime }}=X+\frac{{\left(v_{FV}^{{}^{\prime }}\right)}^2-{\left(v_{FV}\right)}^2}{2a_{FV}}+RD\hfill \end{array}$$

(2)

where $v_{SV}$ and $v_{FV}$ are the initial speed of SV and FV; $v_{SV}^{\prime }$ and $v_{FV}^{{}^{\prime }}$ are the projected speed of SV and FV, respectively; $a_{SV}$ and $a_{FV}$ are the instantaneous acceleration of the SV and FV; $t_{SV}^{\prime }$ represents the time duration for which the FV anticipates that the SV reaches the end of the acceleration lane. $X$ and $X^{\prime }$ are the initial (projected) gap distances between the SV and FV, which are relevant to the traffic volumes; $RD$ is the remaining distance on the acceleration lane for the FV.

Take the merging scenario as an example. If the FV chooses the strategy of accelerating, the aggressive behaviors of both vehicles may lead to emergency braking to avoid a collision. Therefore, the FV requires a hard acceleration $Ac{c}_{MA}$ formulated by the projected states and a limited braking rate, as in Equation (3).

$$Ac{c}_{MA}=minmum\left(\frac{2(X^{{}^{\prime }}-v_{FV}^{{}^{\prime }}{t}_b)}{t_b^2},-4.5{\mathrm{m}/\mathrm{s}}^2\right)$$

(3)

where $t_b$ is the reaction time window of the FV (i.e., 2 s [1]).

The FV can also ignore the SV’s influence and do nothing to keep its following state. However, this leads to two situations: (a) If the remaining gap is $X^{\prime }>0$, the two vehicles still have a potential conflict, so the FV performs as it does in the accelerate-and-merge situation; (b) if $X^{\prime }\le 0$, the FV should have passed the SV, so it can indeed do nothing to keep its state, as Equation (4) shows.

$$Ac{c}_{MDN}=\left\{\begin{array}{cc}Ac{c}_{MA},\hfill & \mathrm{if}{X}^{{}^{\prime }}>0\hfill \\ a_{FV},\hfill & \mathrm{if}{X}^{{}^{\prime }}\le 0\hfill \end{array}\right.$$

(4)

Another strategy for the FV is early deceleration to show a courtesy yield. In this situation, whether the SV chooses to merge or wait, the FV decelerates and gets the payoff $Q_{31}\mathrm{or}{Q}_{32}$. It chooses a comfortable deceleration $Ac{c}_{WDN}$ with the average braking rate (i.e., $-3{\mathrm{m}/\mathrm{s}}^2$ [33]), which is obtained from Equation (5).

$$Ac{c}_{MD}=Ac{c}_{WD}=minimum\left(\frac{v_m-v_l}{t_m^{{}^{\prime }}-t_0},-3{\mathrm{m}/\mathrm{s}}^2\right)$$

(5)

For the scenarios in which the SV chooses to wait for the next available gap, the possible payoff of the FV can be similarly formulated (refer to [1,21] for details).

3.2.2. Payoffs for the SV

The SV has two strategies at the decision time: merging to the mainline and waiting for the next available merging gap. Its payoff functions are also shown in Table 2, and the required acceleration is calculated according to the initial and the projected states.

Consider the situation of accelerating and merging. The SV has to reach the merging point with prior aggressive acceleration $Ac{c}_{max}$ to avoid a collision, as shown in Equation (6). Suppose that the SV accepts the acceleration signal and waits for the next available gap. In that case, the SV calculates the required acceleration according to the remaining distance in the acceleration lane, and the payoff $Ac{c}_{WA}^{\prime }$ can be shown as in Equation (7).

$$\begin{array}{cc}\hfill & t_{MA}=\frac{\sqrt{s_{SV}^2+2Ac{c}_{max}RD}-v_{SV}}{Ac{c}_{max}}\hfill \\ \hfill & Ac{c}_{MA}^{\prime }=\frac{2(RD-v_{SV}{t}_{MA})}{t_{MA}^2}\hfill \end{array}$$

(6)

$$t_{WA}=\frac{(V_{SV}-V_{FV})+\sqrt{{(v_{SV}-v_{FV})}^2+2a_{FV}X}}{a_{FV}}Ac{c}_{WA}^{\prime }=\frac{v_{SV}^{\prime }-v_{SV}}{t_{WA}}$$

(7)

If the FV chooses to decelerate, the SV may receive the courtesy signal from the FV and finish the merging operation with a comfortable acceleration $Ac{c}_{comfort}$, as formulated in Equation (8). The SV can also show a courtesy signal to wait for the next available gap, and the payoff is shown in Equation (9).

$$\begin{array}{cc}\hfill & t_{MD}=\frac{\sqrt{s_{SV}^2+2Ac{c}_{comfort}RD}-v_{SV}}{Ac{c}_{confort}}\hfill \\ \hfill & Ac{c}_{MD}^{\prime }=\frac{2(RD-v_{SV}{t}_{MD})}{t_{MD}^2}\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & t_{WD}=\frac{\sqrt{v_{SV}^2+2Ac{c}_{comfort}(RD-v_{SV}{t}^{\prime }SV)}-v_{SV}}{Ac{c}_{comfort}}\hfill \\ \hfill & Ac{c}_{WD}^{\prime }=\frac{2(RD-v_{SV}(t_{SV}^{\prime }+t_{WD}))}{t_{SV}^2}\hfill \end{array}$$

(9)

Finally, when the FV does nothing to respond to the SV’s lane-changing signal and the SV decides to merge, the SV may have to avoid a collision with the acceleration $Ac{c}_{MDN}^{\prime }$. Thus, the SV has a similar payoff to that in the accelerating situation, that is, $Ac{c}_{MDN}^{\prime }=Ac{c}_{MDN}$. If the SV gives up the merging chance and waits for the next available gap, it calculates the required acceleration $Ac{c}_{WDN}^{\prime }$ with reference to the remaining distance in the acceleration lane. Equation (10) gives the specific formulation.

$$\begin{array}{cc}\hfill & t_{WDN}=\frac{(V_{SV}-V_{FV})+\sqrt{{(v_{SV}-v_{FV})}^2+2a_{FV}X}}{a_{FV}}\hfill \\ \hfill & Ac{c}_{WDN}^{\prime }=\frac{v_{SV}^{\prime }-v_{SV}}{t_{WDN}}\hfill \end{array}$$

(10)

4. Data Sources and Processing

4.1. Data Description

4.1.1. NGSIM Data

This study adopted the celebrated NGSIM dataset [11] for model calibration and validation; it was collected along a section of Interstate 80 in Emeryville, California, in April 2005. The site was about 1650 feet long, and an on-ramp at Powell Street was included. The whole dataset contained three 15 min periods with a total of 200,000 data records (vehicle–seconds) containing detailed information, such as speed, acceleration, gaps, and other variables that are utilized in this study.

Figure 2 illustrates the study site’s lane geometry. The existence of an on-ramp and off-ramp is expected to cause many mandatory lane changes. From the NGSIM database, it can be determined if an on-ramp vehicle is performing a merging maneuver by checking the lane label.

4.1.2. Data Collected from the UAV Aerial Survey

The NGSIM data have been widely used in the lane-changing modeling field because of the large amount and clear marks. However, there has been a growing minority of scholars who have found unrealistic relationships in the NGSIM data [34,35]; some errors are even beyond anything that could be corrected strictly through cleaning or interpolation [36]. Thus, to test the prediction capability of the proposed model framework in different environments with a high accuracy, in this study, a high-resolution UAV aerial photography experiment was designed and conducted to collect the vehicle trajectories in an on-ramp mandatory lane-changing scenario. The data collection site was an on-ramp section of the Xi’an Ring Highway, Xi’an, Shaanxi, China. This site included a four-lane highway section with a metered on-ramp section. The length of the acceleration lane in this investigation was about 200 m. This section was filmed at 25 frames per second with a DJI Phantom 4 Pro hovering at altitudes of 150 m. The survey was conducted from 5:30 to 6:00 p.m. on 25 May 2021. Figure 3 demonstrates the geometry of the investigation site.

In order to mine micro-traffic information hidden behind the video, such as vehicle position, velocity, and acceleration, we proposed a vehicle trajectory processing method based on YOLO [37] and Deepsort [38]. Unfortunately, the official weight set of YOLO could not identify vehicles from the perspective of aerial photography. The transfer learning framework shown in Figure 4 was used to train an effective weight set. A physical image analysis and modeling software named Tracker was also used to finish the vehicle trajectory mining. Notice that, no matter what, there was an observation error between the mining trajectory and true trajectory. This may have been caused by the UAV moving or errors in the detection algorithm. Thus, a two-step straightforward wavelet analysis [39] was used to reconstruct the vehicle trajectories.

4.2. Merging and Non-Merging Identification

This paper defines the vehicle merging process from the moment of lane-changing intention to the final smooth insertion in the target lane. To identify the strategies, the data were cleaned according to the following rules:

• Our study mainly focused on car drivers’ merging and yielding behaviors, so other types of vehicles (i.e., trucks and motorcycles) were removed.
• Ali [1] pointed out that a balance between merging and waiting events is necessary because it will impact the calibration and validation results. A reasonable decision-making horizon should be selected to avoid the dominance of non-merging events. This study empirically extracted the drivers’ strategy based on a 2 s interval.
• The merging process was split into three phases, as in [25], which were the lane-keeping (LK) decision horizon, lane-changing (LC) decision horizon, and LC duration. The point at which the SV passed the lane boundary was regarded as the LC point.
• The collected data were inconsistent in their time periods. A short time period could only partially represent part of the process of vehicle merging; some of the driving behaviors during lane-changing were ignored. Too long of a time period caused an imbalance of merging and non-merging events and amplified the noise in the vehicle trajectory, which was detrimental to the subsequent model training and testing. Therefore, the first 5S data before the LC point were used to identify the merging and non-merging behaviors.

4.3. Decisions of Players in Typical Scenarios

According to the game strategies of both sides, we divided the on-ramp section merging behaviors into six typical scenarios. The SV’s decisions (i.e., merging or waiting) could be directly obtained from the vehicle trajectories, but using a subjective method such as visual observation [21,22] to get the FV’s strategies could cause a significant error. To prevent such errors and biases, this study adopted the Bottom-Up algorithm to extract the lagging vehicle’s strategies. This algorithm was accepted for segmenting trajectory data in the study of [1,24].

The Bottom-Up algorithm segmented the speed profiles and contained segment numbers and corresponding slopes in a matrix. Ozaki [40] proposed the definition of the steady-state regime as Equation (11).

$$S_l=\left\{\begin{array}{cc}accelerating,\hfill & k>0.05\mathrm{g}\hfill \\ donothing,\hfill & -0.05g\le k\le 0.05\mathrm{g}\hfill \\ decelerating,\hfill & k<-0.05\mathrm{g}\hfill \end{array}\right.$$

(11)

where $S_l$ represents the strategy of the lagging vehicle; k represents the corresponding speed slopes; g is the acceleration due to gravity.

Noticing that there are unrealistic velocity and acceleration distributions in the I-80 vehicle trajectory data, which change or even hide the real spatial and temporal relationships. A wavelet-based filter [39] was applied to smooth the vehicle trajectories in this study, and the smoothing preprocessing was performed before any other data analyses.

Combining the merging and non-merging identification and the decision identification rules, a total of 429 lane-changing game scenarios were selected through the NGSIM vehicle trajectories, including 198 lane-changing records and 231 waiting records. In the UAV experiment, 38 lane-changing and 48 waiting records were selected. The distribution of the extracted strategies is shown in

Table 3. The distribution of the extracted strategies.

	SV		FV			Total
	Merge	Wait	Accelerate/Block	Do Nothing	Decelerate/Yield
NGSIM data	198	231	46	331	52	429
UAV data	34	48	12	64	10	86

.

5. Model Calibration and Validation

5.1. Model Calibration

A bi-level programming method was used to estimate the parameters in this paper, which was also used by [1,21,23]. The schematic workflow of the bi-level programming is shown in Figure 5. ${\alpha }_i^k\mathrm{and}{\beta }_i^k$ represent all parameters to be estimated for the SV’s and FV’s payoff functions, respectively; k represents the number of iterations.

The upper level works as a nonlinear programming system with the aim of minimizing the deviation from actual merging actions [21]:

$$min\sum _{i=1}^n[{(F_i-\widehat{F_i}\left(p_{f_i}\right))}^2+{(S_i-\widehat{S_i}\left(p_{s_i}\right))}^2]$$

(12)

where i is an index of an observation; $F_i,\widehat{F_i},S_i,$ and $\widehat{S_i}$ are the observed and predicted actions of the FV (accelerate, decelerate, do nothing) and SV (merge, wait); $p_{fi}$ and $p_{si}$ are the probabilities of the FV’s and SV’s selections, respectively.

The lower level seeks the solution of the Logit Quantal Response Equilibrium. The perception probability of other drivers’ choices is equal to the probability of the drivers’ choices on average, but some errors will occur under the LQRE. The specific probabilities of each strategy are formulated as follows:

$$p_M=\frac{e^{P_{11}·p_A+P_{21}·p_D+P_{31}·p_{DN}}}{e^{P_{11}·p_A+P_{21}·p_D+P_{31}·p_{DN}}+e^{P_{12}·p_A+P_{22}·p_D+P_{32}·p_{DN}}}$$

(13)

$$p_W=1-p_M$$

(14)

$$p_A=\frac{e^{Q_{11}·p_M+Q_{12}·p_W}}{e^{Q_{11}·p_M+Q_{12}·p_w}+e^{Q_{21}·p_M+Q_{22}·p_W}+e^{Q_{31}·p_M+Q_{32}·p_W}}$$

(15)

$$p_{DN}=\frac{e^{Q_{21}·p_M+Q_{22}·p_W}}{e^{Q_{11}·p_M+Q_{12}·p_w}+e^{Q_{21}·p_M+Q_{22}·p_W}+e^{Q_{31}·p_M+Q_{32}·p_W}}$$

(16)

$$p_D=1-p_A-p_{DN}$$

(17)

where $P$ and $Q$ are the expected payoff of the SV and FV defined in Table 2. $p_M$ and $p_W$ are the probabilities of the SV choosing to merge and wait, respectively; $p_A,p_{DN}$ and $p_D$ are the probabilities of the LV choosing to accelerate, do nothing, and decelerate.

All of the probabilities were solved iteratively, and the seed probabilities were used to generate the first set of parameter estimates. When all probabilities converged, the Logit QRE was obtained. Mckelvey [41] proved the existence and uniqueness of the Logit QRE. At this time, the driver’s perception error of the choices followed the extreme value distribution, that is, the drivers had bounded rationality. In contrast, in the Nash equilibrium, the driver expectations were correct and drivers had choices, that is, drivers had absolute rationality.

To obtain the seed probabilities, 300 (of 429) collected samples of all 45 min of reconstructed NGSIM data and 60 (of 86) UAV survey samples were used in the model calibration. By counting the number of different choices, the seed probabilities are given in

Table 4. The seed probabilities of the SV and FV.

	FV			SV
	${\mathit{P}}_{\mathit{A}}$	${\mathit{p}}_{\mathit{DN}}$	${\mathit{p}}_{\mathit{D}}$	${\mathit{p}}_{\mathit{M}}$	${\mathit{p}}_{\mathit{W}}$
NGSIM	0.11	0.72	0.16	0.53	0.47
UAV survey	0.25	0.62	0.13	0.56	0.43

. Mathematically computing the final probabilities was a fixed-point problem, and we adopted the method proposed by [10,41,42] to solve it. It usually took 10 iterations to converge to a fixed point.

Table 5. The calibrated parameters in the NGSIM and UAV surveys.

Strategy	Players	Parameter	NGSIM	UAV Survey	Parameter	NGSIM	UAV Survey
Accelerate and Merge	FV	${\alpha }_{11}^0$	−2.14	2.84	${\alpha }_{11}^1$	2.44	5.40
Decelerate and Merge		${\alpha }_{21}^0$	2.80	−1.31	${\alpha }_{21}^1$	5.90	1.96
Do nothing and Merge		${\alpha }_{31}^0$	0.66	3.10	${\alpha }_{31}^1$	−0.30	−1.31
Accelerate and Wait		${\alpha }_{12}^0$	−1.70	1.35	${\alpha }_{12}^1$	4.24	3.43
Decelerate and Wait		${\alpha }_{22}^0$	5.95	0.36	${\alpha }_{22}^1$	3.12	4.82
Do nothing and Wait		${\alpha }_{32}^0$	2.81	1.78	${\alpha }_{32}^1$	5.92	−1.37
Accelerate and Merge	SV	${\beta }_{11}^0$	3.40	2.74	${\beta }_{11}^1$	−1.18	0.05
Decelerate and Merge		${\beta }_{21}^0$	3.35	1.61	${\beta }_{21}^1$	−1.29	3.09
Do nothing and Merge		${\beta }_{31}^0$	3.78	1.57	${\beta }_{31}^1$	1.57	4.92
Accelerate and Wait		${\beta }_{12}^0$	1.56	0.99	${\beta }_{12}^1$	−2.13	4.70
Decelerate and Wait		${\beta }_{22}^0$	0.48	2.38	${\beta }_{22}^1$	5.30	2.13
Do nothing and Wait		${\beta }_{32}^0$	3.98	−2.90	${\beta }_{32}^1$	−1.39	−1.03

shows the calibrated parameters in the NGSIM and UAV scenarios.

5.2. Model Validation

In this section, the estimated parameters obtained from calibration were used to validate the predictive capabilities of the proposed model for mandatory lane-changing. To assess the model’s performance in detail, we introduced a confusion matrix [1,43]. The matrix consisted of a set of performance indicators that could provide valuable insights for evaluating the effectiveness of the model. The performance indicators that we calculated included: true positive (TP, the prediction matched the observed strategy), false positive (FP, the prediction was merging, while the driver actually took the waiting strategy), detection rate (DR, the percentage of successfully predicted merging behaviors in all merging scenarios), and false-alarm rate (FAR, the percentage of falsely predicted behaviors in all merging scenarios). The specific formulations of the DR and FAR are shown in Equations (18) and (19).

$$DR=\frac{TP}{TP+FP}$$

(18)

$$FAR=\frac{FP}{TP+FP}$$

(19)

The remaining 129 (of 430) interactive cases in the NGSIM and 26 (of 86) interactive cases in the UAV survey were used separately for model validation.

Table 6. The confusion matrix of the model validation results.

	NGSIM I-80					UAV Aerial Survey
	N	TP	FP	DR (%)	FAR (%)	N	TP	FP	DR (%)	FAR (%)
Merge	63	54	9	86	14	14	13	1	93	7
Wait	66	50	16	76	24	12	9	3	75	25
Overall	129	104	25	81	22	26	22	5	85	15

summarizes the validation results and performance indicators of the proposed model. In the NGSIM data, the overall detection rate of the proposed model was $81\%$. It correctly predicted $86\%$ of the merging scenarios and $76\%$ of the waiting scenarios. In the UAV aerial survey data, the overall detection rate was $85\%$, and it successfully predicted $93\%$ of the merging events and $75\%$ of the waiting events. The results imply that the model performed well in predicting the merging interaction on highway on-ramps and showed good predictive capabilities, especially for the merging strategy.

6. Discussion and Conclusions

6.1. Discussion

In the on-ramp sections of highways, the conflicts between a subject vehicle (SV) in the acceleration lane and a following vehicle (FV) in the adjacent mainline lane have received extensive attention in the lane-changing modeling field. Rule-based models and utility-based models describe the SV’s selection with a set of rules or utility functions. However, the influence and response of the FV are ignored. A game-theory-based model incorporates the SV and FV in a competing situation in which one would make a decision under the influence of the other. With the defined payoff functions, a game-theory-based model has strong interpretability and universality, which make up for the deficiencies of artificial-intelligence-based models. Thus, we adopted a game-theory-based approach to modeling the interactions of lane-changing behaviors on highway on-ramps.

In previous studies, most game-theory-based lane-changing models were based on the absolute rational person hypothesis and used the Nash equilibrium solution to predict lane-changing strategies. However, human drivers have perception errors about surrounding traffic conditions and are vulnerable to the influence of previous lane-changing experiences. In this paper, we believe that human drivers determine lane-changing in a bounded rational way, and a game-theory-based approach was used to model the mandatory lane-changing behavior in an on-ramp section. The model comprehensively considered the interaction states of the SV (i.e., merging and waiting) and FV (i.e., accelerating, decelerating, and doing nothing). The bounded rational decision making of the driver was also taken into account with the Logit QRE.

Payoff formulation is an important step in assessing the performance of a developed model. In a mandatory lane-changing interaction, the drivers of the SV and FV need to estimate the future strategy that the other would choose and take the best response strategy. In this paper, the Newtonian equations were used to calculate the projected acceleration payoff in different interaction scenarios. With different strategy sets, the projected acceleration formulation can be quite different. Taking more complex interaction scenarios into consideration may provide some improvements in the model’s performance, which is a topic for future study.

In the calibration and validation section, the vehicle trajectory data from NGSIM and an aerial UAV study were labeled and used separately. The UAV survey data were of a high resolution and precision, and we regarded them as supplementary data for richness and diversity. Note that driving behaviors are totally different in America and China. Certain deviations may have been caused by using the two datasets to calibrate one model. Thus, we calibrated two models with different parameter sets under the modeling framework. The validation results showed that the calibrated two models had good predictive capabilities on their own validation data. That is, the models could perform in multiple environments with parameter re-calibration.

Since there have been some reports on the unrealistic relationships in the NGSIM data [34,35], it was necessary to decrease the influence of the distortion point. We used the wavelet transform to reconstruct the vehicle trajectories, as this was also an important factor restricting the model’s prediction capacity. Note that there were no merging and waiting labels in the vehicle trajectories. Thus, merging and non-merging identification and decision identification methods were proposed to identify the interaction scenarios. We used a 2 s interval window to mine the drivers’ strategies and get a balanced proportion of merging and non-merging events. This was proven to be effective in previous studies [1,25].

To assess the prediction performance of the proposed mandatory lane-changing model, most previous studies used the mean absolute error or root mean square error, which lacked information about model prediction abilities and behavior reliability. Since the confusion matrix is an excellent tool for rigorously and objectively assessing a decision model’s performance [1], we adopted the confusion matrix to evaluate the model’s performance. By calculating the DR and FAR of the proposed model, we found that the model had a good prediction ability, especially for merging events.

6.2. Conclusions

This paper describes a game-theory-based method for modeling the bounded rationality in mandatory lane-changing interactions of an SV and FV in a highway on-ramp section. All of the game’s players made decisions in a rational way, but sometimes made errors according to the probability distribution. The payoff functions were formulated according to the projected acceleration calculated with Newtonian equations. The model calibration was based on bi-level programming, and multiple vehicle trajectory data were used for validation. The model validation results indicated that the model had good predictive capabilities.

Affected by the data conditions, this paper only modeled the vehicle interactions in the on-ramp section of a highway. In the future, the model can be transferred to vehicle trajectory datasets with higher resolutions in different scenes to explore more lane-changing characteristics. The model can also be used in multiplayer game environments. It is of vital importance to model the interactions between multiple drivers for lane-changing decision making, especially for automated vehicles. In addition, a comparison of different types of lane-changing models can also be developed, which will contribute to the understanding of the mechanisms of lane-changing behavior.