Evaluating the Measurement Uncertainty of On-Road NOx Using a Portable Emission Measurement System (PEMS) Based on Real Testing Data in China

Abstract

An evaluation of the measurement uncertainty of on-road NOx emissions using portable emission measurement system (PEMS) based on real local testing data collected in China was carried out as per the type B method defined in the EN 17507 standard. The aim of this work was to quantify the “absolute” measurement uncertainty of PEMSs, which excluded “PEMS relative to laboratory constant volume sampler (CVS)” uncertainty from the calculation of on-road NOx measurement uncertainty using PEMSs. PEMS instruments from three mainstream manufacturers were employed. The zero drift of the NOx analyzers was evaluated periodically during the real driving emissions (RDE) test, and it was noticed that there was neither a linear nor step model of zero drift, with no correlation with the boundary conditions or measurement principle. Additionally, from the 256 valid RDE tests, the zero drift always ranged from 3.8 ppm to −3.8 ppm, and more than 95% of the span drifts were within a range of 1.5%. Based on the laboratory testing of ten vehicles using the worldwide harmonized light-duty vehicle test cycle (WLTC), the type B uncertainty of PEMS NOx measurements corresponding to China-6a and China-6b limits was assessed. An uncertainty of 26.5% for China-6a was found (NOx limit = 60 mg/km over the WLTC), which is very close to the 22.5% from the EU evaluation results (NOx limit = 80 mg/km over the WLTC); the uncertainty with respect to China-6b was found to be 42.8% because the type-I limit was tuned down to 35 mg/km. This result indicates that, with the ever-tightening regulatory limits of vehicle NOx emissions, big challenges will be posed in terms of the reliability of PEMS measurements, which requires PEMS manufacturers to improve the performance of the instruments and policymakers to refine the test procedures and/or result calculation method to minimize the impacts.

1. Introduction

Recently, reducing toxic emissions and carbon footprints and building a more sustainable public mobility system has become unprecedentedly important. Driven by the global development and implementation of ever-tightening vehicular emissions regulations, the measurement and control of on-road emissions from in-service vehicles have become a leading mission for the field, given the expanding discrepancies between lab certification and real driving testing [1,2,3,4,5,6]. This is particularly true after the unveiling of “Diesel gate” in the US in 2015. Driven by this incident, the EU announced the first Commission Regulation, EU 2016/427, in regard to real driving emissions (RDE) control in the first half of 2016, which is more commonly known as “RDE Package 1” [7,8]. Later, at the end of 2016, light-duty China-6 RDE, which largely followed the EU RDE Packages 1, 2, and (the draft of) 3 [9,10], was promulgated.

The introduction of RDE measurement as an in-service compliance program has been shown to be an effective way to purify on-road vehicle emissions, particularly for diesel vehicles. However, the measurement uncertainty for gaseous exhaust pollutants using a state-of-the-art portable emission measurement system (PEMS) has not been fully understood, which has become an obstacle for the surveillance of current legislation and future legislation drafting.

As a pioneer employing PEMSs as a checking tool to prohibit the use of cheating calibration strategies and defeat devices, the accuracy of PEMSs was determined when the not-to-exceed (NTE) requirement was first introduced in the federal regulation of the US [11,12,13,14]. Previous assessment programs were more aimed at the evaluation of the measurement uncertainty of PM rather than NOx. The discrepancies among PEMS instruments using different detection principles were compared by Durbin et al. to a standard constant volume sampler (CVS) [15]. Given that the PEMS was still an emerging technology when these five PEMS instruments were evaluated, inconsistency with the CVS results was yielded from all the units in both steady- and transient-state tests.

Another assessment of PM PEMSs using similar principles was accomplished by the same team a few years later. Refined PEMS instruments performed well in the measurement of engine-out emissions but largely failed in the test program of modern engines with an after-treatment system (ATS) because of the high variation in PM compositions, particularly in the cases of diesel particulate filter (DPF)-equipped engines [16,17]. Similar conclusions were drawn by Mamakos et al., who also noticed the disproportional responses of particle sensors to the nucleation-mode particles discharged within DPF regeneration events, but the performance of the tested sensors during the non-regeneration period was better, and a discrepancy of no higher than 30% was reported in comparison to filter weighing conducted on a heavy-duty chassis dynamometer [18].

A number of PM uncertainty evaluations were also conducted by the European Commission because of increasing concerns over ultrafine particles [19,20,21,22]. However, as an equally important exhaust pollutant and a precursor of secondary particles, the evaluation of the measurement uncertainty of NOx seems scarce compared to PM. Giechaskiel and Valverde et al. systematically reviewed the milestones of the EU RDE regulation and sketched a framework for the assessment of NOx measurement uncertainty using PEMSs [23,24,25,26]. However, those programs were more diesel-oriented, given that the major contributions of NOx and particulate emissions were related to diesel. More recently, the European Committee for Electrotechnical Standardization (CENELEC) included PEMS uncertainty evaluation in their calibration standards and guidelines [27]. Although it includes the type B evaluation of uncertainty, the calculation process is complicated, and the data are not easy to obtain through testing.

As the largest vehicle market in the world, no comprehensive evaluation of PEMS measurement uncertainty has been conducted in China thus far. Although China has largely followed the EU RDE Package 1 to 3, it is not feasible to quote the evaluation results of the EU because of regulation details and limited values. For example, China-6 regulations set “fuel-neutral” emissions limits for petrol and diesel vehicles, as well as more stringent NOx limit values than Euro-6d, i.e., NOx for the M₁ category at room temperature, 35 mg/km vs. 60/80 mg/km. China-6 regulations permit the use of market-available petrol for a proper RDE test, which means, in most cases, petrol without ethanol substitution (E0) will be used, which is different from the prevalent E10 in EU member states. In addition, the requirements of ambient conditions, testing altitude, and driving dynamics are also different.

All these differences lead to the necessity to conduct an evaluation of the on-road measurement uncertainty of PEMSs based on the China-6 RDE regulations, especially before the RDE regulations are formally enforce on 1 July 2023.

In this work, the first evaluation of NOx measurement uncertainty using a PEMS was conducted based on local testing data collected in China; meanwhile, the evaluation method for the analyzer zero drift and span drift was refined on the basis of the European framework. Additionally, it should be mentioned that the uncertainty determined in this paper is the “absolute” measurement uncertainty of the PEMS, a concept different from the “relative to CVS” uncertainty quantified by Giechaskiel and Valverde et al. [25,26]. This paper aimed to identify the source of uncertainty related to each RDE test procedure and help PEMS manufacturers and RDE policymakers understand the future challenges of implementing more stringent on-road emissions testing regulations.

2. Materials and Methods

2.1. Uncertainty Framework

The framework is based on the EU framework, with the exception of excluding the relative uncertainty of the standard CVS system, which can be seen in Figure 1.

The EU framework was developed to evaluate the “relative uncertainty”, the additional measurement uncertainty of the PEMS relative to the laboratory constant volume sampler (CVS), also defined as conformity factors (CF) in the European regulations. This paper aimed at the estimation of the “absolute” measurement uncertainty of state-of-the-art PEMS instruments. Thus, CVS bag uncertainty was not considered in the scheme shown in Figure 1 since, as previously mentioned, all factors were taken into account with the error propagation rule.

2.2. Scopes of Uncertainty Evaluation

In this framework, distance uncertainty, analyzer accuracy, gas accuracy, and analyzer linearity are determined by the requirements of regulations. Additionally, some factors have been evaluated reasonably by the JRC (Joint Research Centre), such as EFM uncertainty and time alignment/dynamics, which are indicated in gray in Figure 1; therefore, we used the evaluation results of the JRC. The above parameters are beyond the scope of this paper. The NOx emission limit was updated to the China-6b limit. The main aim of this paper was to re-evaluate the analyzer span drift and zero drift. Moreover, the impact of the moving average window (MAW) method was beyond the scope of this paper since the MAW method is not representative [28].

2.3. Data Source for Evaluation

The test data employed for the uncertainty evaluation in this paper came from the voluntary RDE testing program organized by the Vehicle Emission Control Center of the Ministry of Ecology and Environment of China from 2016 to 2018. More than 100 manufacturers, engineers, experts, and tier 1 PEMS suppliers participated in this annual event. A total of 54 China-6 certified/prototype vehicles were tested in the lab or on the road. Further, 223 valid RDE test results were collected from 10 cities with varying ambient temperature and humidity, altitude, road slopes, and extents of congestion. From 2019 to the time of this study, our research team conducted 33 valid RDE tests on 11 China-6 vehicles to ensure the timeliness of the experimental data.

Otherwise, some tests were specifically designed for the understanding of PEMS uncertainty. In total, 14 zero/span drift validation results and 10 WLTC results were summarized as the input of this study.

2.4. PEMS Instruments

In this paper, the three most commonly used models of RDE-compliant PEMS instruments were adopted for evaluation in case of any bias induced by a specific detection principle. For each model, more than one kit was employed in the test campaign, which minimized the artifact error brought by the malfunction of any single unit. A total of 11 instruments were used for testing to minimize possible malfunction-induced errors. In general, the testing principles for NOx emissions from light-duty vehicles are Chemiluminescence (CLS) and Non-Dispersive Ultraviolet (NDUV). Because the NDUV method is an optical method based on the spectrum of vehicle exhausts, NO and NO₂ emissions are identified and quantified simultaneously. No additional calculation is required, which helps reduce the extra uncertainty produced in the process of data calculation. However, in light of the fact that water produces significant interference in the spectrum of NOx species, caution should be exercised regarding the inaccuracy of PEMS instruments using the NDUV principle.

2.5. Test Vehicles

In order to make the evaluation result more representative of the market, the test vehicles covered a wide range of engine displacements, from 1.0 to 3.0 L, identical to those in Kousoulidou et al. [3], Giechaskiel et al. [23,25] and Valverde et al. [26]. The testing results from a sum of 65 China-6 certified/prototype vehicles were obtained for this evaluation. The test vehicles were acquired from domestic and overseas manufacturers. In addition to popular cars and SUVs, light-duty vans were also considered in this work. There were both conventional internal combustion engine (ICE) and hybrid models in the test fleet. For petrol engines, models using multi-point port fuel injection (PFI), direct injection (GDI), and combined injection strategies were considered. Generally, these vehicles provide broader coverage of the models available in the market and better representativeness. The main specifications of the test vehicles are summarized in

Table 1. Vehicle information.

Category	Vehicle Information
Vehicle manufacturer	Domestic, overseas
Propulsion system	ICE, hybrid
Injection system	PFI, GDI
Fuel	Gasoline, diesel
Emission stage	China-6
Vehicle type	Passenger car, SUV, light-duty truck
Displacement	1.0–3.0 L
Intake system	Naturally aspirated, turbocharged
Transmission	MT, AT, DCT, CVT
Drive mode	Front-wheel drive, rear-wheel drive, four-wheel drive

.

2.6. Testing Conditions

In order to quantify the uncertainty of NOx measurement using a standard PEMS in China, the tests were performed in ten cities with various altitudes and ambient temperatures and humidities. The details of the testing boundary conditions in each city are listed in

Table 2. Boundary conditions.

City	Test Date	Altitude (m)	Ambient Temperature (°C)	Ambient Humidity (%)	Ambient Pressure (kPa)	Cumulative Positive Elevation Gain (m/100 km)
Kunshan	Aug. 2017	6.4	25.7	31.8	102.0	161.2
	Oct. 2018	7.2	22.6	59.1	101.8	173.8
Xiamen	Oct. 2018	14.0	27.4	47.8	102.3	375.6
	Jun. 2021	23.6	31.8	57.5	100.9	400.1
	Mar. 2019	16.1	29.5	59.3	101.0	345.3
	Aug. 2021	22.9	27.2	48.1	99.9	418.1
	Sept. 2021	19.6	23.0	52.0	100.8	417.5
	Oct. 2021	22.6	25.7	42.0	101.6	390.9
Beijing	Aug. 2017	27.9	34.0	52.2	100.5	260.4
	Nov. 2018	27.6	18.9	30.7	102.6	253.0
	May 2020	32.5	28.5	34.8	100.2	226.6
Qionghai	Oct. 2018	32.5	33.1	54.5	101.1	347.6
Xiangyang	Sept. 2018	88.9	36.2	27.8	100.2	370.1
Chongqing	Oct. 2017	285.8	16.9	99.4	98.7	436.3
	Sept. 2018	409.8	22.9	69.9	97.2	944.6
	May 2020	387.0	22.2	72.1	97.3	856.3
Yinchuan	Sept. 2017	1112.8	20.6	50.8	89.2	215.8
Lanzhou	Sept. 2017	1592.2	17.7	74.2	84.6	476.3
Kunming	Oct. 2017	1896.1	18.7	72.9	81.8	406.0
	Sept. 2018	1956.7	21.9	61.1	80.4	680.6
	Jun. 2020	2039.6	23.6	56.1	79.3	788.1
Xining	Sept. 2017	2257.9	18.4	46.9	77.9	358.0

.

3. Results and Discussion

3.1. Zero Drift

In the 2017 evaluation conducted by the JRC team, two assumptions, namely, linear drift and step drift, were adopted for the estimation of PEMS zero drift, and the permissible limit of 5 ppm for zero drift was used [25]. Undoubtedly, this method is scientifically sound and logically reasonable, but the research team also pointed out that in future work, the actual performance of PEMS instruments should be evaluated on a proper RDE route, which would substantially improve the practicability of the results. In the JRC 2018–2019 report [26], a special case of significant drift was selected for the zero-drift evaluation. The China RDE regulation-making group realized this issue in 2016 and accumulated test data under extreme temperature, humidity, and altitude conditions from 2016 to 2021, which were used in the evaluation reported in this paper.

In this study, the zero drift of the three examined PEMS models was determined on various valid RDE test routes in different cities under different boundary conditions. The PEMS units were first calibrated using zero and span gas prior to the start of testing. During the test, the zero points of the NO/NOx analyzers were checked for 5 min manually every 15 min of operation. This zero-checking procedure was repeated 4–6 times for each RDE trip, and the readings of the NO/NOx analyzers were recorded to express the zero drift during real RDE testing. The average of each 5 min checking window of each PEMS model is plotted in Figure 2 as a function of time.

Figure 2 plots the zero-drift performance of the PEMS instruments from three main manufacturers on valid RDE routes. It can be observed in Figure 2 that the performance of PEMS #1 was steady and reliable, even in Kunming, a city with a very high testing altitude and ambient humidity. The actual zero drift of PEMS #1 was always less than 1 ppm. There are no signs showing that the zero drift of PEMS #1 was influenced by the boundary conditions. PEMS #2 showed a different tendency than PEMS #1. In the first half of the tests, the zero drift of PEMS #2 seemed to be even larger than in the second half, which strongly disobeys the concept of linear drift. Nevertheless, the poorest zero drift of PEMS #2 was about 3 ppm, about 40% smaller than the permissible limit of 5 ppm in the current regulations. The rest of the zero reading points were all within 2 ppm. It is interesting to note that in three of the four testing cities, PEMS #3 performed well, and the majority of the zero-drift checks were maintained at a level of <1 ppm. However, during the test in Kunshan, where the testing altitude, temperature, and humidity were the closest to laboratory conditions, the zero drift of PEMS #3 jumped to −3.4 ppm unexpectedly 20 min after the start of testing, and in the next check, the zero drift was preserved around −4 ppm. Nonetheless, the zero drift of PEMS #3 could still fulfill the legislative requirement for values no larger than 5 ppm. Because there was no malfunction alarm during this test, it is unknown what type of flaw induced this phenomenon.

Overall, the PEMSs showed virtually no correlation with the boundary conditions or measurement principle. The zero drifts detected in real testing practices seem more like spontaneous incidents. Using either a linear or step model to estimate the uncertainty attributed to zero drift could result in a considerable error. Compared to the linear assumption, step drift is more conservative because it assumes the maximum drift happens right after the start of testing, which always overestimates the uncertainty along with the zero drift. Additionally, the method presented in the JRC 2018–2019 report, i.e., picking a specific case, does not seem to be representative of the selection of significant drift cases to assess zero drift. Therefore, we still think that step drift is a more reasonable choice for now.

Figure 3 summarizes the frequencies of the 256 recorded zero-drift check events in the valid RDE tests, along with the 95% confidence window. It can be seen that, in most cases, the zero drift of the PEMS NOx detector ranged from −3.8 ppm to 3.8 ppm, while in the conclusion of the JRC 2020 report, 3 ppm is the reported value. Based on this fact, in the calculation of the additional NOx emissions corresponding to zero drift, this paper used 3.8 ppm zero drift to replace the permissible limit of 5 ppm used in the JRC 2017–2019 report and China-6 regulations.

Based on the WLTC, Figure 4 plots the results of the additional NOx emissions calculation using 3.8 ppm drift as input. In this figure, the additional NOx emissions were still calculated using linear and step drift assumptions separately since, according to Figure 2, the zero drift in the RDE measurement behaved like a combination of these two regimes. It can be seen in Figure 4 that over both the WLTC and its low-speed stage, the additional NOx emissions derived from the step drift assumption were always more than 50% higher than those from the linear assumption. Over the WLTC, the maximum additional NOx emissions calculated using the step drift assumption were 9.4 mg/km. The value for the low-speed stage of the WLTC was 14.5 mg/km.

It can also be observed in Figure 4 that neither the additional NOx emissions over the WLTC nor its low-speed stage showed a clear inclination with the physical displacement of the engines. This is due to the employment of turbocharging and hybrid propulsion technologies. In Figure 5 there is an evident correlation between the exhaust flow rate and additional NOx emissions. Figure 5 also helps to identify the worst case of zero drift among the ten test vehicles, which was about 5.1 mg/km.

3.2. Span Drift

Similar to Figure 3, Figure 6 summarizes the 256 span drift check events from the valid RDE tests. It was found that more than 95% of the span drifts were within a range from −1.5% to 1.5%. Hence, in this paper, a span drift level of 1.5% was used to replace the 2.0% level that the JRC team used in their 2017–2020 report [23,26,29].

In agreement with the JRC’s assumption of span drift, the span drift was assumed to be linear, with 0% at 0 ppm and 1.5% at the span concentration. We took 1.5% of the span drift into the calculation of additional NOx emissions over the WLTC and its low-speed stage. The results are illustrated in Figure 7. It can be noticed that the influence of span drift at such a level on the final NOx calculation was almost negligible. Even over the low-speed stage of the WLTC, which is the most sensitive stage to span drift, the derived additional NOx emissions were only 0.027 mg/km or 0.027% in percentage, far smaller than the impact of zero drift.

Using the uncertainty framework, Figure 8 presents the combined uncertainty of PEMS NOx measurements using local data collected in China and China-6 limits as input. Corresponding to the 60 mg/km and 35 mg/km limits set for China-6a (effective nationwide from January 2021, delayed due to COVID-19) and China-6b (effective nationwide from July 2023), the calculated PEMS NOx measurement absolute uncertainties are 26.5% and 42.8%, respectively.

When taking into account the deduction in the CVS uncertainty, the PEMS NOx measurement relative uncertainties of 6a and 6b are still 26.5% and 42.8%, respectively. Compared to the EU 2017 uncertainties of 24.7% (linear zero drift) and 43.5% (step zero drift) [23], the EU 2018–2019 uncertainty of 32.0% [26] and the 2021 published re-evaluation result of 22.5% [29], the uncertainties derived from the Chinese regulation are generally larger, which is predominantly due to the more stringent limit values. In the three rounds of uncertainty evaluation, the JRC team targeted diesel vehicles and thus used the 80 mg/km NOx cap. However, China-6 is a fuel- and technology-neutral regulation, which means that the uniform limit values are applicable to vehicles in a certain category using any fuel or technology.

4. Conclusions and Future Prospects

To better support the promulgation of RDE regulations in the current and future vehicle emission regulations in both China and the EU, a systematic evaluation of the NOx measurement uncertainty using PEMSs was carried out based on real testing data collected in China. In addition to the “relative to CVS” uncertainty of PEMSs evaluated by the JRC, this paper also considered the uncertainty of a CVS. The main findings of this paper are presented below.

(1) Zero-drift levels were checked in real-world RDE testing practices under a series of extreme boundary conditions to replace the permissible limit of 5 ppm used in the previous evaluation. The new method checked the zero points by pumping pure nitrogen every 15 min under standard operating conditions. In a broader range of ambient temperatures, humidities, and altitudes, 95% of the detected zero drifts were observed to be within 3.8 ppm. The largest additional NOx emissions for 1.5 to 3.0 L engines due to a step zero drift of 3.8 ppm were 9.4 mg/km, with the worst case being 5.1 mg/km.

(2) In total, 95% of the span drift events were found to be within 1.5%, and the evaluation using this value showed that span drift only posed a negligible impact on the final calculation of NOx emissions.

(3) The uncertainties of PEMS NOx measurement corresponding to China-6a and China-6b limits were 26.5% and 42.8%, respectively, which are larger than the EU’s 2017 and 2020 results, mainly because of the more stringent NOx limits of the China-6 regulations.

The results of this paper also indicate that, along with the implementation of stricter NOx emission limits, serious challenges will be posed in terms of the reliability of PEMS measurements, which requires PEMS manufacturers to improve the performance of instruments and policymakers to refine the test procedures and/or result calculation methods to minimize the impacts.

A deficiency of this paper is that the current assessment of NOx PEMS measurement uncertainty is predominantly based on gasoline-fueled light-duty passenger cars. To make the assessment results more representative, more diesel-fueled light-duty vehicles will be included in future assessments.

In addition to NOx, the on-road measurement uncertainty of CO, solid particles, and CO₂, considering the global need for carbon neutrality, needs to be sufficiently understood to create a solid basis for future regulation promulgation.