An Effective Two-Stage Algorithm for the Bid Generation Problem in the Transportation Service Market

Abstract

This study designs a two-stage algorithm to address the bid generation problem of carriers when adding new vehicle routes in the presence of the existing vehicle routes to provide transportation service. To obtain the best auction combination and bid price of the carrier, a hybrid integer nonlinear programming model is introduced. According to the characteristics of the problem, a set of two-stage hybrid algorithms is proposed, innovatively integrating block coding within a genetic algorithm framework with a depth-first search approach. This integration effectively manages routing constraints, enhancing the algorithm’s efficiency. The block coding and each route serve as decision variables in the set partition formula, enabling a comprehensive exploration of potential solutions. After a simulation-based analysis, the algorithm has been comprehensively validated analytically and empirically. The improvement of this research lies in the effectiveness of the proposed algorithm, i.e., the ability to handle a broader range of problem scales with less time in addressing complex operator bid generation in combinatorial auctions.

1. Introduction

In the transportation service market, carriers act as bidders and face a classic bid generation problem (BGP) in determining their optimal bidding strategies. When bidders fail to align their bids accurately with the actual demand for transportation services, it can lead to either underutilization or overcapacity in the transportation network. This waste not only increases operational costs but also contributes to environmental inefficiencies, making it a pressing concern. Meanwhile, carriers in the real world usually make bidding strategies in complex scenarios and need to bid on combinations of items, such as both prices and vehicle routes, rather than individual items. To overcome the above challenge, scholars are increasingly focusing on the combination auction in the bid generation problem, which can efficiently match the shipper’s transportation needs with the carrier’s services and reduce logistics time and geographical restrictions, leading to improved logistics efficiency.

Combination auction, which refers to a process where the shipper’s transport needs are matched with carrier transport services through the simultaneous auction of routes and prices, has been defined in research by Smith et al. [1] and Blumrosen et al. [2]. It is conducive to cross the inconvenience of geography and time, and reduces the collection of information and participation, as demonstrated by Abrache et al. [3]. Palacios et al. [4] surveyed the practical applications of combinatorial auctions, emphasizing their pivotal representational and economic facets. They simplify the information gathering and negotiation process for shippers, reduce information and transaction costs, and help carriers flexibly adjust their strategies to maximize returns, while also optimizing resource allocation. These mechanisms can flexibly deal with the uncertainty of the logistics market so that shippers and carriers can better adapt to market changes.

To address the bid generation problem, existing algorithms designed often fall short due to various limitations. Some algorithms may lack the capability to effectively handle the intricate combinations of transportation services, resulting in suboptimal bidding strategies [5,6,7,8]. Others, while potentially effective, may be computationally intensive, rendering them impractical for large-scale problems within reasonable timeframes [5,8]. Therefore, it is necessary to design a more effective algorithm.

Based on the above background, this paper studies the carrier’s bid generation problem on both optimal bidding routes and bidding prices by designing a two-stage algorithm. We first construct the carrier’s existing transport network and auction route into a multi-graph network, then propose a mixed-integer nonlinear programming model. According to the characteristics of the model, we propose a two-stage algorithm by selecting a genetic algorithm (GA) as the underlying algorithm framework and embedding a depth-first search (DFS) algorithm, i.e., DFS-GA. By taking the auction price as the decision variable, the scale of the example improves, and the performance of the algorithm increases over 2 times.

The contribution of this research is twofold. First, in the existing literature on combination auctions, some studies focus on the collaborative calculation of route combination [5] or only the collaborative calculation of auction price to reduce the empty load rate [6,9]. In contrast, this paper considers the collaborative calculation of auction route combinations and auction prices at the same time.

Second, we propose an innovative two-stage hybrid algorithm and make significant improvements to the encoding scheme utilized in our algorithm. This enhanced encoding allows us to represent bid combinations more efficiently. Compared with results obtained by Gurobi (GA and improved PSO), our algorithm offers much faster runtimes, scalability to larger problems, and improved solution quality.

The rest of this study is organized as follows: we review the relevant literature in Section 2, Section 3 illustrates a hybrid integer nonlinear programming model, Section 4 designs a two-stage algorithm to solve the above model, Section 5 discusses numerical results with numerical examples, and Section 6 concludes the study and gives future research directions.

2. Literature Review

In the past years, the decision problem of bidding prices for transportation service portfolio auctions has been a complex task that requires careful consideration of various factors. To address this challenge, by considering both winning probability and conditional profit, Tan et al. [10] broadened standard auction models by introducing the Cobb–Douglas function, enabling a more intricate evaluation of bidder preferences. Owing to the great complexity of BGPs in nature, most research has considered various factors such as capacity [7], auction game strategy [8], route optimization and collaborative transportation [5,6,9], bi-level programming and bundle generation problems (BuGP) [11,12], the online combinatorial auction (CA) [13,14], truckload (TL) transportation [5,15,16,17], and fourth-party logistics (4PL) [18,19].

Due to the inherent complexity of BGPs, the researchers took into account key factors such as vehicle capacity and flow balance. Yan et al. [7] introduced an analytical framework designed to detect profitable transportation task bundles. Auction game strategy and route optimization are equally important when determining bid prices. Yuan et al. [8] proposed a two-phase model. In the initial phase of this model, a pricing strategy was developed that is optimized for both large-scale and small-scale carriers, taking the complexities of a stochastic bid generation problem into account.

Cooperative transportation has been a hot topic in recent years, especially in three-tier transportation networks with time windows. Lyu et al. [5] constructed a mathematical programming model rooted in path analysis to address the bid generation dilemma in the combinatorial auction for procurement of transportation services, considering time windows within a three-tier transportation network; The issue of carrier bid generation in collaborative transportation was delved into by Mamaghani et al. [6], utilizing a combinatorial auction. Additionally, a multi-period bid generation approach encompassing two distinct types of pickup and delivery requests was introduced; Karels et al. [9] examine an auction approach that facilitates carrier collaboration without compromising the autonomy of individual entities. Multiple auction instantiations are investigated.

Bi-level programming and bundle generation problems have been emerging research directions in transportation service auctions in recent years. An extensive analysis of the bidding generation problem was conducted by Yan et al. [11], focusing specifically on utilizing a bi-level programming framework to delve into its intricacies; the bundle generation problem, as defined by Gansterer et al. [12], involves the objective of delivering a minimized selection of bundles while optimizing the total coalition profit, ensuring a practical allocation of bundles to carriers.

Online combinatorial auctions are of particular importance when serving customers in transportation networks. The online combinatorial auction, serving customers stationed at nodes of a transportation network and emphasizing carbon emissions, was the focus of Triki et al. [13]. This auction framework facilitates timely shipments for carriers, optimizing load consolidation and minimizing repositioning excursions. Furthermore, the study considers sustainability and carbon emissions by prioritizing carriers’ carbon reduction efficiencies; by focusing on combinatorial auctions, Hammami et al. [14] explore a negotiation framework that benefits shippers by outsourcing transportation operations and carriers by gaining new transportation contracts.

In truckload transportation, vehicles with no load will be empty when they travel to the next destination to load goods. Too high an empty load rate will increase the transportation cost of the carrier. Therefore, some studies have focused on the synergy of auction routes to reduce the empty load rate of transportation, make reasonable use of transportation resources, minimize the transportation expenses incurred by the carrier, and maximize the revenue generated by the enterprise. Within a multi-round combinatorial auction setting for truckload transportation service procurement, the carrier’s bid generation challenge is being addressed. Lyu et al. [15] consider multiple timeframes and the uncertainty surrounding requests to optimize the carrier’s anticipated net earnings. This involves carefully choosing transportation requests, service periods, and routing options for both reserved and additional requests. Utilizing location techniques, Triki [16] crafted an optimization approach aimed at bolstering the synergistic effects between auctioned bundle loads and the intricate interaction between auctioned and pre-existing loads; utilizing the central limit theorem (CLT), Zhang et al. [17] propose a unique data-driven approach for fashioning polyhedral uncertainty sets. Their methodology incorporates two distinct scenarios: shipment volumes varying independently, as well as those exhibiting correlated patterns. Additionally, acquiring truckload transportation services via combinatorial bidding processes poses three problems, which are addressed by Lyu [5] in his research.

From a fourth-party logistics provider’s point of view, Qian et al. [18] explore a revised winner determination problem that incorporates the risks of bidder disruptions in the procurement of transportation services by a fourth-party logistics provider through combinatorial reverse auctions. To alleviate these risks, a novel two-phase stochastic winner determination model is introduced, incorporating a comprehensive hybrid mitigation approach that encompasses reinforcement, reservation, and external option strategies. Furthermore, drawing upon a multifaceted decision-making approach, Qian et al. [19] formulated linear constraints tailored to each bidder’s sustainability and flexibility metrics. Subsequently, they seamlessly integrated an external option policy into their framework, developing a sophisticated two-phase stochastic model that accurately determines sustainable and flexible winners.

In conclusion, while our research shares similar objectives, auction mechanism applications, and mathematical modeling techniques with the reviewed literature, it differs in terms of network construction, algorithm design, and model validation. We propose the construction of a multi-graph network integrating the carrier’s existing transport network and auction routes. We introduce a two-stage collaborative algorithm combining the depth-first search and the genetic algorithm, with auction price as the decision variable.

3. Problem Description

Combinatorial auction is an effective means to minimize shipper costs and maximize carrier profit. As a result, in the auction process, the carrier should consider the integration of its own existing transport network into the auction route, and then make the auction decision. Therefore, this section will show how to find an efficient way to integrate existing transport routes and auction routes by designing a two-stage algorithm, and at the same time, derive the auction price by considering the synergy between the routes.

$$\mathrm{M}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e} R+{\sum }_{i\in I}{\sum }_{{(j,k)}^2\in A_i}{p}_{{(j,k)}^2}^i{x}_{{(j,k)}^e}^i-{\sum }_{i=1}^{I}v{d}_i{t}_i{y}_i$$

(1)

where $R$ denotes the revenue obtained from the existing routes; $I$ denotes the number of routes; $A_i$ denotes lane include in route $i$; $p_{{(j,k)}^2}^i$ denotes bidding price for auctioned lane (j, k) in route $i$; $x_{{(j,k)}^e}^i$ is a random variable that obeys binary distribution, i.e., $x_{{(j,k)}^e}^i=1$ if node $j$ to node $k$ with lane of type $e$ in route $i$ is chosen to be submitted, otherwise $x_{{(j,k)}^e}^i=0$; $v$ denotes unit distance cost for a truck; $d_i$ denotes the distance of route $i$; $t_i$ denotes the number of trucks in route $i$; $y_i$ is a random variable that obeys binary distribution, i.e., $y_i=1$ if route $i$ is chosen to be submitted, otherwise $y_i=0$.

Equation (1) indicates the maximum profit value of the carrier. It should satisfy the following constraints:

$${\sum }_{i=1}^I{y}_i{t}_i\le T,$$

(2)

$$M{y}_i\ge {\sum }_{{(j,k)}^1\in A_i}{x}_{{(j,k)}^1}^i+{\sum }_{{(j,k)}^2\in A_i}{x}_{{(j,k)}^2}^i \forall i\in I,$$

(3)

$$y_i\le {\sum }_{{(j,k)}^1\in A_i}{x}_{{(j,k)}^1}^i+{\sum }_{{(j,k)}^2\in A_i}{x}_{{(j,k)}^{2 }}^i\forall i\in I,$$

(4)

$${\sum }_i^I{x}_{{(j,k)}^2}^i\le 1 \forall {(j,k)}^2\in A_i,$$

(5)

$$d_i{y}_i\le \omega \forall i\in I,$$

(6)

Equation (2) denotes that the carrying capacity of the carrier’s auction route shall not exceed its own carrying capacity. Equations (3) and (4) make sure each transport route meets specific requirements. Equation (5) denotes the auction channel can be covered at most once in a submitted package. Equation (6) makes sure each route must satisfy the length constraint.

4. Solution Methodology

This problem is a bid combination problem in path optimization. It is an NP-hard problem with a two-stage model of two-level programming. Many numerical experiments prove that genetic algorithms can efficiently solve multi-vehicle unstable dynamic time window planning problems and pick-up and delivery path optimization problems. Considering that this problem is also a path optimization problem, the genetic algorithm (GA) is used as the basic optimization algorithm framework. However, because the auction combination problem has too many factors to consider and belongs to the large example BGP model, the basic genetic algorithm will lead to problems such as too large a scale of computing power, bloated initial coding, and easily falling into local optimality. In the case of too many routes, the running time will be too long, and the optimal solution of satisfactory results will not be obtained.

To solve this problem, we propose a two-stage hybrid algorithm that combines both a depth-first search algorithm and a genetic algorithm. Figure 1 illustrates it vividly. In the first stage, a depth-first search algorithm (DFS) is used to enumerate all possible transport path sets I according to relevant path constraints, and the decision variable $y_i$ is defined to indicate whether the transport path $i$ is selected. In the second stage, the improved GA iterative solution will be used to determine the auctioned routes in the bid mix and the corresponding offers, and finally, the maximum return route will be solved.

4.1. Depth-First Search Algorithm (Phase 1)

A depth-first search algorithm is used to exhaustively search the tree, traversing the nodes of the tree one by one. When a new node is discovered, the untraversed adjacent edge is probed. If all edges are explored, the search will be traced back to the starting node of the edge where the node v was found. The whole process repeats until all nodes are accessed. The algorithm procedure can be divided into four steps:

Step 1: According to the line set, generate the carrier’s multidigraph transportation network with the corresponding demand and distance.

Step 2: Starting from the initial parking lot, enter the DFS algorithm loop.

Step 3: Iterate into the next node of the line, meanwhile judging whether the path conditions by Yang et al. [20] are met. If not, return to the previous line loop and repeat this step. It should be noted that this constraint is attached to the maximum route freight vehicle limit $T.$

Step 4: Obtain the path set. Import the synergy table and calculate the correlation coefficient to get the synergies $s_{j,k}^i$.

4.2. Improved Genetic Algorithm (Phase II)

The second stage will use an improved GA iterative solution to determine the auctioned lines in the bid mix and the corresponding offers. To solve the problems mentioned above, this paper chooses to reconstruct the encoding and decoding modes, optimize the roulette selection, and improve the crossover and mutation modes, to improve the code running speed and avoid falling into the local optimal solution.

For all the generated qualified routes, this paper first encodes them in real numbers, and then groups all the generated paths according to existing service routes to meet the constraints and improve and reduce the redundancy of the algorithm. Then, the chromosome length is defined as the number of existing service routes, and each gene length is an index value corresponding to the grouping capacity of existing service routes. Its genotype is the index value of the group corresponding to the selected route. On the contrary, if you decode it, pay special attention to the identification of two identical paths, and only select one.

Taking the model of 7 cities, 7 existing service lines, and 9 auction lines as an example, which is also given in Figure 2, a total of 28 lines can be obtained by the DFS algorithm, which is numbered from 0 in turn. First, the 28 routes are grouped into 7 existing service routes and can be encoded according to the above coding rules. As shown in Figure 2, the chromosome number is “3221534,” and the decoding operation is carried out on it. The first 3 represents the 0 group of routes whose index is 3, namely 8, which is highlighted in light color; the second place indicates a set of routes with index 2, i.e., 15; and similarly, 7 routes of 8, 15, 3, 11, 16, 21, and 26 are selected last.

The roulette selection algorithm in the common genetic algorithm needs to calculate the cumulative probability of each parent, but this will cause the algorithm scale to become large and the convergence speed very low, and it is easy to lose the global optimal solution and fall into the local optimal situation. Based on the unique decoding process, this paper combines tournament selection with roulette, so that each generation of optimal solutions flows directly into the next generation, and only the first 50% of the parent generation of roulette selection, to give individuals with low fitness more room for evolution.

To prevent rapid convergence without finding the global optimal solution, this paper carries out a unique cross-mutation design based on its unique coding method. Among them, the crossover operation introduces the calculation factor count, and after the two-point crossover, the calculation factor that meets the chromosome length is randomly defined for multi-point crossover again to ensure the diversity of genotypes. As shown in Figure 3, the first step is a two-point crossover, and the second step is a multi-point crossover, which are represented in two colors. Then, an additional swap of two nodes follows, and its resulting offspring is represented by a third color.

The variation operation is carried out in two ways: directional variation and non-directional variation according to the priority of its corresponding route. Undirected variation is to randomly generate count 1 to carry out the corresponding number of chromosome variations, and the variation method uses the random function to randomly select each gene length. The convergence variation is to randomly generate count 2, select count 2 gene fragments in the chromosome, and convert them into the gene index with the greatest difference, set the maximum length corresponding to each gene fragment as size, the original gene as $o$, and then the index calculation formula is SIZE-($o$+1). The specific process is shown in Figure 4. Deep blue represents the initial genetic factor, light blue represents the uncertainty mutation factor, and blue represents the deterministic mutation factor.

5. Numerical Experiments

In this section, we conduct all computational experiments on a HUAWEI PC (Intel Core I 7 2.92 GHz) with 16 GB RAM and Windows 11 v23H2(22631.3296). Furthermore, the program codes are written in Java. To test the performance of the designed hybrid algorithm, we adopt Gurobi 10. 0 and the heuristic algorithm designed by other scholars to construct several comparison experiments.

Since it is difficult to obtain real-time data on auction combinations, we conducted simulation experiments based on the intercity transportation situation in China and previous research of scholars.

5.1. Configuration of Test Problems

In previous studies, the size of the examples was 9, 12, 15, 18, 20, 22, 25, 28, 30, and 35, respectively. The reason for setting the data this way is to compare them with the previous paper by Yang [20], which used only a precise algorithm. Now, we use a heuristic algorithm to solve the actual larger-scale problem. In this paper, the maximum example is increased to 80 after the introduction of the improved algorithm. The specific parameters of the calculation example are shown in

Table 1. Initial setting parameters.

Case	N	Booked Lanes’ Quantity	Auctioned Lanes’ s Quantity	Empty Lanes’ s Quantity
1	7	7	9	14
2	10	12	27	18
3	15	22	38	30
4	20	23	49	40
5	25	28	53	50
6	30	33	73	60
7	40	47	81	80
8	60	52	139	120
9	80	67	187	160

below.

The first column denotes nine data experiments with different scales conducted in this study. The “N” represents the number of cities in the example, while the last three columns represent the number of booked, auctioned, and empty lanes, respectively. In addition, the quantities of booked and auctioned lanes are generated by simulation based on a realistic real-order situation, while the empty lanes’ quantity is 2N, since empty lanes exist only between depot 0 and nodes.

5.2. Computational Results

This section compares the results obtained by using Gurobi and the two-stage algorithm to verify the quality of the solution, because the solution obtained by Gurobi is the optimal solution after traversing each feasible solution satisfying the constraints. All of the numerical experimental results in this chapter are the average values obtained after running the program 20 times, and the results that run for more than one hour are considered invalid and displayed in the table with “-”.

In

Table 2. The comparison between Gurobi and DFS-GA.

N	Gurobi			DFS-GA
	Route	Obj	Time	Route	Obj	Time
7	6	14,155.2	1.12 s	6	14,155.2	0.33 s
10	15	23,166.3	4.65 s	15	23,166.3	2.25 s
15	35	51,443.4	433.45 s	35	51,443.4	6.67 s
20	51	131,673.2	355.67 s	51	131,673.2	8.99 s
25	67	143,311.1	500.02 s	67	143,311.1	11.09 s
30	-	-	-	71	147,873.2	28.90 s
40	-	-	-	68	164,533.8	47.89 s
60	-	-	-	57	152,626.7	70.21 s
80	-	-	-	63	132,323.6	77.23 s

, when the data scale is small, Gurobi can obtain the optimal solution of the model in a short time, but with an increase in the number of nodes, the model solution time increases rapidly. When the number of cities exceeds 25, Gurobi cannot get the optimal solution in a finite time. It is found that the results of the two-stage algorithm are consistent with those of the Gurobi solution in small-scale examples. In contrast, the operation time of the DFS-GA algorithm has been reduced by about 50 times when N is 25, which greatly improves the performance of the algorithm.

The Obj term, the profit of the model, peaks when N is 40 and decreases as N continues to increase. The reason for this phenomenon is that the maximum number of vehicles and the longest travel distance are set in the model; that is, when the model reaches the maximum capacity, the transportation task, the revenue constant penalty function, and the transportation cost continue to increase, which leads to the total profit becoming smaller. In addition, most of the time, this two-stage algorithm is used to generate feasible routes, and the feasible routes do not change much after the maximum capacity is reached, so the running time of the example after the maximum capacity is reached does not increase much.

To further investigate the performance of the improved algorithm, we compare the two-stage algorithm designed in this paper with the traditional GA algorithm and the improved particle swarm optimization (PSO) algorithm designed by Yan et al. [7] for the average calculation results of 30 runs. The meaning of each index in

Table 3. Performance comparison between DFS-GA and other algorithms.

N	DFS-GA			GA			Improved PSO
	Route	Obj	Time	Route	Obj	Time	Route	Obj	Time
7	6	14,155.2	0.33 s	7	12,685.7	8.33 s	7	14,155.2	0.23 s
10	15	23,166.3	2.25 s	17	20,321.5	152.45 s	15	22,874.2	1.98 s
15	35	51,443.4	6.67 s	42	42,435.6	605.67 s	35	46,583.5	7.78 s
20	51	131,673.2	8.99 s	58	106,533.9	1718.4 s	43	101,332.8	10.01 s
25	67	143,311.1	11.09 s	-	-	-	54	127,541.3	11.23 s
30	71	147,873.2	28.90 s	-	-	-	59	115,832.4	30.11 s
40	68	164,533.8	47.89 s	-	-	-	45	98,323.1	50.19 s
60	57	152,626.7	70.21 s	-	-	-	52	72,336.6	99.67 s
80	63	132,323.6	77.23 s	-	-	-	61	86,372.9	140.21 s

is consistent with that in Table 2. It can be seen in Table 3 that with the expansion of computing power, the traditional GA algorithm cannot enter the algorithm iteration in time or seek a better solution. As for the improved PSO algorithm, it is slightly faster than the algorithm designed in this paper at a small scale, but the final value is far lower than for the two-stage algorithm. In the largest example, the DFS-GA algorithm has a higher profit than the previous algorithm, and the running time is reduced by 50%. In other cases, the performance of all comparison indexes of the two-stage algorithm exceeds that of the two algorithms. By comparison, the designed algorithm can get better route recommendations in different scales of examples, and the adaptability is stronger than that of the previous algorithms.

5.3. Sensitivity Analysis

When analyzing the sensitivity of carrier profit to the change of each parameter, Example 1 is used to ensure the sensitivity of the data, applying the two-stage algorithm to solve the problem (the operation time is less than 1 s).

Figure 5 shows the sensitivity of path length ($\omega$), unit transportation cost ($\upsilon$), and number of carrier vehicles ($T$). These three factors are tested separately, and these three factors are represented by blue, red, and green lines, respectively, and adjusted by −20%, −10%, +10%, and +20% on the original value, then brought into the BGP expansion model to solve and obtain the broken line change diagram in the figure. Analyzing the image, it is found that as $\upsilon$ increases, the maximum profit decreases continuously, while the other two factors are the opposite. Secondly, the changes of $\upsilon$ and $T$ have a large impact on profit. However, when $T$ is reduced within 10%, it does not have any effect on the experimental results. Finally, it is found that a $\omega$ reduction of 0–10% has a great impact on profits, while the sensitivity of the other cases is weak.

In the process of pursuing profit maximization, a carrier can seek measures to reduce unit service costs and increase its profit; extending drivers’ working hours can also increase profits. In the transportation service network, the capacity of some vehicles is a waste of resources, so it is necessary to reasonably plan the connection relationship between vehicle transportation tasks.

6. Conclusions

The primary focus of our article lies in the development and application of a hybrid algorithm that effectively addresses the complexities of the vehicle routing problem within the context of combinatorial auctions. Since we designed the auction combination model without the matching algorithm in the previous research, based on this model, this paper designs a set of two-stage hybrid algorithms. This greatly improves the scale of the example, and the performance of the algorithm increases over 2 times. The core idea is to make use of the existing service route in the path generation condition. We group the route first and then encode it, meanwhile reconstructing the cross-mutation operation. The algorithm designed in this paper has been certified by Gurobi results and compared with other algorithms designed by other scholars. The results are good.

In this paper, 2896 paths are considered under the background of the 414 paths mentioned above. The calculation results show the model can obtain the global optimal solution better. The first challenge of the algorithm is to obtain all feasible solutions satisfying the constraint in a very short time; the other is to resolve real-time practical problems, because data processing takes plenty of time. Experimental results show that, compared with the Gurobi optimization algorithm, the two-stage hybrid algorithm DFS-GA has a faster computation speed. DFS-GA significantly reduces solution time, especially for larger-scale instances where the number of cities (N) increases. The efficiency improvement factor ranges from about 3.4 for smaller instances to more than 40 and even more than 65, indicating a significant speedup.

There are three possible research directions in the future. One is to further improve the heuristic algorithm to generate routes and solve models faster. The other one is to carry out reinforcement learning through large numbers of data sets for simultaneous decision-making; of course, they can also be combined to settle larger-scale practical problems.