Evaluation Index for IVIS Integration Test under a Closed Condition Based on the Analytic Hierarchy Process

Abstract

The intelligent vehicle infrastructure system (IVIS) requires systematic testing before being put into large-scale applications. IVIS testing under closed conditions includes stress tests for typical scenarios and extreme scenario strength testing. To extract IVIS integration test indicators under closed conditions, this article constructed a hierarchical framework of IVIS’s evaluation indexes in the stress tests and the strength tests. The hierarchical framework of IVIS stress test evaluation indicators reflect the highway construction area under typical scenarios, and the hierarchical framework of IVIS strength test evaluation indicators reflect the highway merging area under extreme scenarios. Both are based on the test requirements of the stress test and strength test, with safety as the evaluation objective. Second, the analytic hierarchy process (AHP) was used to calculate the weights of the test evaluation indicators of the two scenarios. Finally, the activity-based classification (ABC) method was used after ranking the weight results in order to extract the key factors that have the maximum impact on safety in the scenarios. In this paper, we proved the practicality and feasibility of the AHP-ABC extraction method in the IVIS integration testing evaluation index and guided the development and testing of the IVIS.

1. Introduction

The intelligent vehicle infrastructure system (IVIS) is expected to be one of the best safety solutions towards fully automatic driving. However, before being put into large-scale applications, IVIS should be tested systematically [1]. IVIS test evaluation can be divided into two categories in general, one for traffic operation and one for connected and automated vehicles (CAVs) working conditions. There has been a large amount of work on traffic operation evaluation methods based on travel time [2], road capacity [3], velocity [4], and traffic flow [5]. Studies on CAV working condition evaluation methods were according to components [6,7] and functions [8]. At the same time, many closed CAVs test (CAVTest) fields have realized key test elements for CAV functions. Taking the closed CAVTest of Chang’an University in China [9] as an example, the critical test elements generally include a variety of road environments (cities, highways, and rural roads), environmental complexity (weather, emergencies), road infrastructure (road signs and markings, traffic control facilities, wading roads), auxiliary test equipment (dynamic and static obstacles), and network conditions (RSU, communication protocols, lidars). However, the traffic operating and CAV working conditions are inextricably linked, and a single evaluation for any side will inevitably fall into the problem because of the light evaluation mechanism. In other words, there are two main problems with the existing studies of IVIS key test elements. On the one hand, the current studies are conducted according to the working conditions of each system. However, IVIS is not only a simple superposition of road systems and CAV functions but also an organic fusion of traffic engineering, information technology, and the automotive industry, having the characteristics of multisystem cooperation and with specific functions corresponding to particular needs. On the other hand, in the current closed CAVTest fields, the test scenario is generally for a single refined scenario, such as the communication function test for LET-V, and AEB for the pedestrian test [9]. The IVIS technology in mass production faces comprehensive and complex scenarios, such as the high-speed on-ramp merging scenario, which involves communication, cut-in, obstacle recognition, and reaction and other functions, which have seldom been discussed in most of the existing studies.

In IVIS, according to the degree of openness, the system can be divided into closed and semi-open conditions. Due to article space limitations, we only focused on the selections of the IVIS integration test evaluation index under closed conditions. The closed condition refers to the conditions in which the IVIS elements are repeatable and steerable or are not affected by external factors [9]. As shown in Figure 1, IVIS testing under closed conditions should be executed in two parts: one is to meet the stress test (subdivided into industry development demand, technical test demand, and response testing demand), while the other is to match the strength test (subdivided into extreme weather testing demand, extreme road conditions demand, and extreme communication test demand) [10,11]. Then, the demand–indicator matching will extract typical scenario indicators for the stress test and extreme scenario indicators for the strength test. This IVIS testing framework of evaluation indexes can realize comprehensive testing in typical and extreme scenarios according to testing demand.

In general, to address the lack of evaluation for specific requirements and integrated evaluation in the above studies, we refined the testing requirements under IVIS integration testing in closed conditions, constructed a testing index framework for IVIS in closed conditions, and used the AHP to establish an evaluation framework. To reflect the importance of different indicators, we obtained the weight of each indicator through an expert survey [12] and finally evaluated the IVIS integration test indicators using activity-based classification taxonomy (ABC taxonomy, also known as the Pareto method).

The remainder is organized as follows: Section 2 reviews the previous works, Section 3introduces the evaluation methodology, Section 4 analyzes the arithmetic examples, and Section 5 concludes this study.

2. Literature Review

IVIS is divided into traffic operation and CAV working conditions. In terms of traffic operation evaluations, they are diverged into traffic operation efficiency evaluations and safety evaluations at different scales, such as road networks and intersections. The efficiency evaluations are generally according to travel time [1], driver behavior and market penetration [13], vulnerability (extreme weather impact within the region) [14], and speed [15]; safety evaluations are generally according to speed [16], time to collision (TTC) [17,18], and economic and environmental efficiency evaluations [19,20]. In terms of evaluating CAV working conditions, the testing is generally based on vehicle components or functions separately, such as brake pedal evaluation [6,7], crash testing [8,21], and even communication delay [22,23] or communication safety [24], as well as on the overall CAV performance [25].

In the study of evaluation methods, there are economic benefits assessment analysis and multiobjective analysis methods for the evaluation of traffic operations. The economic benefits assessment analysis method usually evaluates the socioeconomic convenience of transportation projects through cost–benefit analysis [26]. Multiobjective analysis methods include the analytic hierarchy process [20,27,28]; fuzzy comprehensive evaluation [29]; the Delphi method [30]; and the combination method of economic benefit analysis, the multi-criteria method [31,32], and multiobjective analysis [13]. On the other hand, the evaluation of the CAV working conditions is divided into simulation experiment methods [6,7], the analytic hierarchy process [12], and fuzzy comprehensive evaluation [25].

In conjunction with the above study, numerous studies in the field of transportation have been conducted using the AHP method and have proven their effectiveness, such as the evaluation of the impact of environmentally friendly transportation measures on urban sustainability [33], the evaluation of the safety impact of road projects on bridges [34], the evaluation of the current state of the transportation system in Beijing [35], the decision to deploy electric vehicle charging infrastructure [36], the evaluation of hinge factors for the selection of electric vehicle charging stations [37], and the evaluation of the primary occupational hazards of road construction workers [38].

Other studies verified the usefulness of Pareto analysis (aka the activity-based classification method, abbreviated as the ABC method) in identifying the majority of critical factors that play a decisive role. For example, total quality management (TQM) is often used to analyze the crucial factors for the success of total quality management in companies [39,40,41] or to analyze the crucial factors of food safety assurance systems [42]. Therefore, the ABC method has an excellent performance in the study of key factors and is applicable to the screening of key indicators for IVIS.

The distribution of the literature for current IVIS evaluation studies is shown in

Table 1. Review of current IVIS evaluation studies.

Current IVIS Evaluation Studies
Traffic Operation		CAVs Working Conditions
Efficiency Evaluations	Safety Evaluations	Vehicle Components or Functions
Travel time [1]	Speed [16]	brake pedal evaluation [6,7]
Driver behavior and market penetration [13]	TTC [17,18]	crash testing [8,21]
Vulnerability [14]	economic and environmental efficiency evaluations [19,20]	communication delay [22,23]
Speed [15]		communication safety [24]
		the overall CAV performance [25]

. From Table 1, we can see that most of the studies are relatively homogeneous, studying only one index to evaluate the traffic operation or the CAV working conditions. However, IVIS is a coupled human–vehicle–environment system, and a single index cannot satisfy the evaluation of one or more scenarios operating under IVIS. In addition, AHP in combination with other methods has been shown to be effective [43,44]; thus, in this paper, the AHP weighting results of IVIS are used for the screening of key indicators using the ABC method, and from there we propose a method that enables a comprehensive evaluation of the IVIS operation in different scenarios in a complex IVIS environment, and such a comprehensive evaluation can be executed with fewer evaluation indicators, which can guarantee the reliability of the evaluation results and improve efficiency.

3. Methods

In this paper, the AHP method was used to construct judgment matrices for the highway’s construction area stress test and merging area strength test layer by layer; the eigenvectors of each judgment matrix were obtained in order to find the weight results of each layer and the final weight results of each index. Finally, according to the final weighting results classifying the ABC method for key index screening, we obtained the indicators to be tested for different scenarios of the IVIS system under closed conditions. The framework of this proposed method is shown in Figure 2.

3.1. Demand–Indicator Matching for Scenarios

Demand–indicator matching requires upper-layer framework construction, lower-layer indicator extraction, and demand–indicators matching based on scenario selection of the following three steps:

• Step 1: Upper layer framework construction

The hierarchical structure of IVIS evaluation under closed conditions generally consists of four layers, which are the target layer, criterion layer, subcriterion layer, and indicator layer from top to bottom. Since [9] points out that the IVIS test conditions under closed conditions are controllable and that IVIS easily achieves optimal efficiency, we only thought over the establishment of the hierarchical structure when the objective was secure. As the network cloud has become a crucial component of the IVIS, we introduced a “cloud” based on the three scales of “human–vehicle–road”, as shown in Figure 3. In Figure 3, “A“ represents the total objective of the target layer, and Security A represents the total objective A in the hierarchical analysis is to evaluate the IVIS security; “B” represents each component in the criterion layer, which is composed of four scales of human–vehicle–road–cloud after decomposition according to the coupling characteristics of IVIS; “C” represents each component in the sub-criterion layer, which is a sub-criterion of different dimensions after further refinement on the basis of the criterion layer B, for example, to distinguish vehicle B2 by single vehicle C3 and fleet C4. Moreover, the impact of on-cloud firewalls and on-cloud databases on the central cloud in the cloud criterion was considered. From here, the upper framework was established.

2.

Step 2: Lower layer indicator extraction

Combining the literature research in the Literature Review section, we focused on the three aspects of multiple layers (target layer, criterion layer, subcriterion layer, and indicator layer), multiple scales (human, vehicle, road, and cloud), and multiple dimensions (IVIS system, infrastructure, and traffic participants including people and vehicles) and established the indicator set of the IVIS, as shown in Figure 4.

Specifically, traffic operation, travel time, congestion level, and TTC are usually used as optimization objectives in the evaluation, and those optimization objectives are generally calculated on the basis of the three major parameters of traffic (traffic volume, speed, and density)—these parameters can be transformed into each other as well, and thus the index system covers traffic volume, speed, travel time, saturation, etc. Driver behavior, vulnerability, and the uniqueness of the scenario also need to be considered, and thus the indexes also cover factors such as the driver’s mental state, construction area protection measures, number of conflict points, and extreme weather. CAV testing often is divided into functional testing of a single vehicle and cooperative testing of a cruising fleet. Since CAV technology relies on network cloud implementation, targets related to vehicle communication and network connectivity are considered in the set. In addition, the vehicle position, acceleration, and TTC are also physical quantities describing the vehicle and fleet motion state [45] and are controlled by the vehicle speed and travel time. Thus, vehicle speed and travel time are also part of the evaluation indicator set instead of position, acceleration, and TTC. Thereby, some indicators cross each other in the traffic operation evaluation and CAV testing, which are combined in the indicator set and shown only once, as shown in the overlapping part. The indicates in the overlapping part between the sets are called “common features”.

3.

Step 3: Demand–indicator matching based on scenario selection

Existing studies use intersections as the closed test field [1,46] because of the complexity. However, when it comes to using the test results for large-scale applications, the index results obtained from the tests are not well adapted to the actual application requirements. This is because the randomness of human and vehicle behavior in urban intersections exceeds the testing capabilities available in closed test sites (e.g., high density and large bursts of humans and vehicles cannot be well reproduced). In other words, the intersection is more reasonably classified as a test scenario under semi-open conditions.

The integration test of IVIS under closed conditions mainly includes the stress test and the strength test. The stress test tests the maximum load (stable and peak) that the system can achieve under certain load conditions by continuously increasing the system load. The impact of the system caused by the test object of the stress test is within the rated operating limits of the system and does not cause irreversible damage to the system. The strength test means that under atypical operating conditions, the test object’s influence on the system is beyond the system’s rated operating limits to measure the system’s operating condition under overload conditions.

In our study, in order to exclude the influence of highly random scenario elements on the test results, a stress test was conducted under closed conditions that was based on a repeatable cut-in test on a highway scenario [17], specifically a construction area with frequent vehicle cut-ins, as well as a strength test based on on-ramp merging, which can achieve high efficiency [47]. As shown in Figure 5, the CAV on lane 1 has to cut into lane 2 because of the construction area in front. The highway construction area is a rare scenario in highways with the participation of non-drivers and traffic practitioners for operational activities, which means a certain degree of freedom for humans, and its uncertainty affects the operation of the construction area. As shown in Figure 6, when the car on the ramp is ready to enter the main road, it will intertwine with lane 1 and lane 2 on the main road to generate at least two conflict points, and the higher the traffic flow, the more conflicts there are. The on-ramp merging area is the main conflict area where traffic intertwines on the highway, and thus we used the merging zone as a scenario to construct a safety test index system under extreme conditions.

Therefore, we took the highway construction area as the typical scenario for the stress test and the on-ramp merging area as the limit scenario for the strength test. Then, the stress test requirements and strength test requirements were matched with the above framework of the IVIS under closed conditions in order to obtain the corresponding indicators of the indicator layer under the above different scenarios. The framework of the IVIS highway construction area stress test evaluation index system under closed conditions is shown in Figure 7, and the framework of the IVIS on-ramp merging area strength test evaluation index system is shown in Figure 8.

The frameworks shown in Figure 7 and Figure 8 are both complete hierarchical frameworks. This complete hierarchical framework was formed by the metrics in Figure 4 after matching with the upper hierarchical framework for different scenarios and distributing them in subsets of the subcriteria layer to form a feature layer, and this feature layer was combined with the upper hierarchical framework. Moreover, since the features in certain sets were not applicable in different scenarios, the subcriteria and indicators included in different scenarios were different. For example, as shown in Figure 8, there will be no traffic practitioners in the highway merging area, and thus there was no subcriteria C1 and its subordinate indicators. For another example, the highway merging area shown in Figure 8 considers the test indicators in the strength test, which had certain requirements for extreme conditions, and therefore the impact of natural disasters (D24) needs to be considered, while the highway construction zone shown in Figure 7 was in the state of pressure test, and therefore D24 did not need to be considered. On the other hand, “common features” coexisted in many sets, and there were duplicate matches in the matching process between the upper hierarchical framework and the lower criteria, and thus there were duplicate indicators in the complete hierarchical structure of Figure 7 and Figure 8, such as D2, D3, and D4, i.e., common features caused the duplication of indicators in Figure 7 and Figure 8.

In summary, in the case of the upper hierarchy common to all scenarios in the IVIS environment obtained through IVIS feature decomposition and refinement, similarities (common upper hierarchy framework) and differences (different indicators due to different operating conditions (e.g., traffic practitioner/non-practitioner, or stress/strength testing)) appeared between Figure 7 and Figure 8.

Then, a two-by-two comparison judgment matrix for each level of Figure 7 and Figure 8 above can be obtained from the expert evaluation questionnaire. The weights of each criterion can be gained by calculating the maximum eigenvalue and the corresponding eigenvalue from the matrix. Finally, in line with the weights of each level and indicator, the total weights of each indicator in the target level were obtained.

3.2. Weight Calculation Based on the AHP Method

For the hierarchical analysis structure established in Figure 7 and Figure 8 for different scenarios, we invited several experts to score the importance of each index from 1 to 9 [12,20] and construct a judgment matrix. The scoring was based on the five-level scale importance, together with the corresponding meanings, which are listed in

Table 2. Judgment matrix scale and meaning.

Level of Importance	Definition (Element i with Respect to Element j)
1	Equally important
3	Slightly more important
5	Significantly more important
7	Strongly more important
9	Extremely more important
Countdown	Inverse value of element i over element j

.

Then, the eigenvectors $W$ of the judgment matrix can be obtained, and the consistency test $CI$ of the judgment matrix based on the eigenvectors $W$ was as shown in the following equation:

$$CI=\frac{{\lambda }_{\max }-n}{n-1},$$

(1)

where ${\lambda }_{\max }$ is the maximum eigenvalue of the judgment matrix and $n$ is the order of the judgment matrix. When the judgment matrix order is greater than 2, the ratio of the judgment matrix consistency index $CI$ to the average consistency index $RI$ [20] is calculated as the random consistency ratio $CR$. When $CR<0.10$, the judgment matrix has acceptable consistency to obtain the index importance ranking in the IVIS integrated test evaluation index system under closed conditions.

3.3. Index Screening Based on the ABC Method

In the IVIS test evaluation index system, the evaluation index for testing under different scenarios may be constrained by economic costs, test equipment, and other constraints, and thus the evaluation index needs to be screened. After obtaining the importance ranking of the indicators in the IVIS integration test evaluation index system under closed conditions, we used the ABC method to identify key factors that are in small amounts but decisive [48]. To filter for the most important evaluation indicators, we classified the indicators obtained from the above hierarchical analysis measurement into ABC categories and constructed an ABC indicator analysis table.

4. Example Analysis of Scenarios

Since most of the functions of the IVIS are still in the research stage, researchers and experimenters have a more specialized understanding of IVIS, and therefore the 10 experts invited for the established hierarchy in this paper were all scholars from research institutions, university professors, and general researchers in related fields. After the comprehensive judgment matrix was obtained from the experts’ scoring table by taking the plural, the matrices that did not meet the consistency test were adjusted, and the overall ranking and weight values of the judgment matrix were calculated. The final judgment matrices all conformed to the consistency test.

4.1. The Stress Test of the Highway Construction Area

In outlined in this section, the importance of the indicators for the evaluation of the IVIS highway construction area stress test under closed conditions was calculated according to the IVIS highway construction area stress test evaluation system, as shown in Figure 7. Taking the human criterion with safety as the target under closed conditions as an example, the judgment matrix and weights of each indicator were calculated as shown in

Table 3. B1 judgment matrix and weights in the construction area.

Criteria Layer	Subcriteria Layer	Indicator Layer	D1	D2	D3	D4	D5	D6	Indicator Weight on the Subcriteria Layer	Subcriteria Weight on the Criterion Layer	Criteria Weight on the Target Layer
B1	C1	D1	1	1	1/3	1/5			0.102	0.5	0.406
		D2	1	1	1/3	1/3			0.116
		D3	3	3	1	1/3			0.264
		D4	5	3	3	1			0.519
	C2	D2		1	1/3	1/3	3	1/3	0.056	0.5
		D3		3	1	1/3	5	1	0.286
		D4		3	3	1	3	1	0.107
		D5		1/3	1/5	1/3	1	1/5	0.229
		D6		3	1	1	5	1	0.321

, which were in the indicator layer under the human criterion for the importance of the safety impact of IVIS; the results of the weight hierarchy of each indicator of human criterion in the construction area were calculated as shown in

Table 4. B1 hierarchical weight results in the construction area.

Indicators	Subcriteria Weight on the Criterion Layer		The Indicator Total Weight on the Criterion Layer
	C1	C2
	0.500	0.500
Construction area traffic volume D1	0.102		0.051
Construction area single-vehicle speed D2	0.116	0.107	0.111
Construction area number of conflict points D3	0.264	0.229	0.247
Construction area safety measures D4	0.519	0.321	0.420
Driver’s driving age D5		0.056	0.028
Driver’s mental state D6		0.286	0.143

.

Due to space limitations, all judgment matrices cannot be listed here. The total hierarchical weight results of indicators in the target layer in the construction area are shown in

Table 5. Total hierarchical weight results of indicators in the target layer in the construction area.

Indicators	B1	B2	B3	B4	Total Weight of Indicators	Sort
	0.406	0.357	0.080	0.157
Construction area traffic volume D1	0.051				0.021	11
Construction area single-vehicle speed D2	0.111	0.112			0.085	3
Construction area number of conflict points D3	0.247	0.072			0.126	2
Construction area safety measures D4	0.420	0.139	0.750		0.280	1
Driver’s driving age D5	0.028				0.011	16
Driver’s mental state D6	0.143				0.058	7
OBU communication time delay D7		0.139			0.050	8
Construction area average single-vehicle travel time D8		0.037			0.013	15
Construction area total average queue length D9		0.042			0.015	13
Construction area total average travel time D10		0.064			0.023	10
Construction area average saturation D11		0.042			0.015	14
Construction area total average speed D12		0.138			0.049	9
V2V communication delay D13		0.214			0.076	4
Construction area equipment sensing and identification delay D14			0.125		0.010	17
Construction area RSU communication delay D15			0.125		0.010	18
Central cloud network communication security protection D16				0.375	0.059	6
Central cloud network communication delay D17				0.125	0.020	12
Central cloud data security protection D18				0.500	0.063	5

. From Table 5, it can be seen that among the stress test evaluation index system of the construction area of IVIS under closed conditions with safety as the target, the construction area safety measure D4 was the most significant factor affecting the overall safety of the construction area, followed by the construction area number of conflict points D3, while the construction area equipment sensing and identification delay D14 and the construction area RSU communication delay D15 were the least influential factors. The above results show that even though IVIS adopted advanced communication technology to assist road operation, communication was not enough to be the main factor affecting the system safety of IVIS. Ultimately, the safety protection of the construction area itself during the road construction period and the regularity of the travel routes in and around the area were the fundamental factors to avoid road risks in the highway construction area. This result may be slightly beyond the general perception; in fact, it is most likely related to the lack of trust in the current IVIS by the experts invited to the survey, which may change with the development of more accurate vehicle positioning and more instantaneous communication technologies.

According to the results of the AHP analysis of the abovementioned safety objectives for the stress test of the construction area, the index weight values were ordered and then accumulated, and the ABC index analysis diagram was drawn, as shown in Figure 9. As seen from Figure 9, the construction area safety measures D4, the construction area number of conflict points D3, and the vehicle–vehicle (V2V) communication delay D13 were A-type indicators, and the above indicators must be included in the test scope when testing and evaluating the IVIS for the construction area under closed conditions; the central cloud data security protection D18, the central cloud network communication security protection D16, the driver’s mental state D6, the OBU communication time delay D7, and the construction area total average speed D12 were B-type indicators, and when testing for construction area scenarios, one can consider whether such indicators need to be included in the testing scope according to the actual testing conditions and testing costs, as appropriate, while the remaining category C indicators can be tested without the need for a more comprehensive assessment of the security of the construction area stress test to save testing costs.

4.2. The Strength Test of the Highway Merging Area

In this section, we calculated the importance of the strength test evaluation indexes of the IVIS highway merging area under closed conditions according to Figure 8. Then, the total hierarchical weight results of indicators in the target layer in the merging area were as shown in

Table 6. Total hierarchical weight results of indicators in the target layer in the merging area.

Indicators	B1	B2	B3	B4	Total Weight of Indicators	Sort
	0.371	0.371	0.094	0.163
Merging area single-vehicle speed D2	0.429	0.073			0.186	1
Driver’s driving age D5	0.143				0.053	6
Driver’s mental state D6	0.429				0.159	2
OBU communication time delay D7		0.051			0.019	14
Merging area average single-vehicle travel time D8		0.027			0.010	21
Merging area total average queue length D9		0.037			0.014	18
Merging area total average travel time D10		0.048			0.018	15
Merging area average saturation D11		0.037			0.014	19
Merging area total average speed D12		0.121			0.045	8
V2V communication delay D13		0.084			0.031	12
Merging area equipment sensing and identification delay D14			0.081		0.008	22
Merging area RSU communication delay D15			0.047		0.004	23
Central cloud network communication security protection D16				0.375	0.061	5
Central cloud network communication delay D17				0.125	0.020	13
Central cloud data security protection D18				0.500	0.081	3
Weather recognition accuracy of OBU D19		0.106			0.039	9
Road condition recognition accuracy of OBU D20		0.106			0.039	10
Forward obstacle recognition accuracy D21		0.138			0.051	7
Lane change recognition accuracy in front D22		0.172			0.064	4
Communication delay of RSU by electromagnetic interference D23			0.186		0.017	16
Communication delay of RSU by natural disasters D24			0.186		0.017	17
Road obstacle warning D25			0.375		0.035	11
Road condition recognition accuracy of RSU D26			0.125		0.012	20

.

As seen from Table 6, among the strength test index system of the merging area of IVIS under closed conditions with safety as the goal, the merging area single-vehicle speed D2 was the most significant factor affecting the overall safety of the merging area, followed by the driver’s mental state D6, while the merging area equipment sensing and identification delay D14 and the merging area RSU communication delay D15 were the least influential factors. The above results indicate that similar to the stress test index of the construction area, the influence of communication equipment on safety was not the most important, and the dangerous driving behavior (single-vehicle speed and driver’s mental state) in the merging section was the fundamental factor affecting safety in the strength test. This could be because, in the strength test in the merging area, the identification and response of the existing IVIS to extreme weather, extreme road conditions, and extreme communication only lies in detecting and sending information, but not yet well enough to perfectly handle with them, and thus the safety of the strength test in the merging area depends more on human driving behavior.

On the basis of the evaluation index weights and rankings shown in Table 6, the analysis chart of ABC indexes of the merging area under closed conditions was drawn, as shown in Figure 10. As seen from Figure 10, the merging area single-vehicle speed D2, the driver’s mental state D6, the central cloud data security protection D18, the lane change recognition accuracy in front D22, and the central cloud network communication security protection D16 were A-class indicators, and the above indicators must be included in the test scope when testing and evaluating for IVIS in the merging area under closed conditions; the driver’s driving age D5, the forward obstacle recognition accuracy D21, the merging area total average speed D12, the weather recognition accuracy of OBU D19, the road condition recognition accuracy of OBU D20, the road obstacle warning D25, and the V2V communication delay D13 were B-type indicators that can be considered according to the actual test conditions and test cost when testing for merging area scenarios. The remaining category C indicators can be tested without the need for a more comprehensive assessment of the security of the construction area pressure test to save testing costs.

5. Conclusions

In this paper, we conducted a demand analysis of IVIS stress tests and strength tests under closed conditions, and also constructed a test index framework around three aspects of multiple layers (target layer, criterion layer, subcriterion layer, and indicator layer), multiple scales (people, vehicle, road, and cloud), and multiple dimensions (IVIS system, infrastructure, and traffic participants), using AHP to conduct a study on the evaluation index system of IVIS stress tests under the scenarios of highway construction areas and IVIS strength tests under the scenarios of highway merging areas.

First, we constructed an IVIS test evaluation index screening method under closed conditions; then, we analyzed the stress test evaluation indexes on the basis of highway construction areas and strength test evaluation indexes based on highway merging areas; finally, we obtained classification indicators for different scenarios of the IVIS integration test under closed conditions, which helped in improving the extraction method of existing IVIS integration test indicators for comprehensive evaluation. The examples showed that those indicators related to driving behavior always reflect a more important influence on safety in different scenarios. The AHP-ABC extraction method also provides advantages for the intelligent networked environment and driving safety. In the intelligent networked environment, the indicators extracted by the method guide smart vehicles and smart infrastructure to certain technology development, and in driving safety, the extraction method can provide full coverage, high-security simulation testing, and verification indicators to ensure driving safety.

IVIS is constantly evolving, and the indicators and scenarios involved in this article may not be exhaustively considered due to the limitations of thinking. It should be noted in particular that due to the limited number of experts, the test indicators, screening methods, and test scenarios mentioned in this article can be further enhanced according to future technology development or richer actual scenarios. For example, we can discuss how to extract the test indicators for curved driving scenarios under closed conditions or further discuss the index evaluation system for typical scenarios of IVIS integration tests under semi-open conditions. The test index evaluation for intersection sites under semi-open conditions will be undertaken in future research.