Development of the Global to Mesoscale Air Quality Forecast and Analysis System (GMAF) and Its Application to PM_2.5 Forecast in Korea

Abstract

This paper presents the development of the global to mesoscale air quality forecast and analysis system (GMAF) and its application to particulate matter under 2.5 μm (PM_2.5) forecast in Korea. The GMAF combined a mesoscale model with a global data assimilation system by the grid nudging based four-dimensional data assimilation (FDDA). The grid nudging based FDDA developed for weather forecast and analysis was extended to air quality forecast and analysis for the first time as an alternative to data assimilation of surface monitoring data. The below cloud scavenging module and the secondary organic formation module of the community multiscale air quality model (CMAQ) were modified and subsequently verified by comparing with the PM speciation observation from the PM supersite. The observation data collected from the criteria air pollutant monitoring networks in Korea were used to evaluate forecast performance of GMAF for the year of 2016. The GMAF showed good performance in forecasting the daily mean PM_2.5 concentrations at Seoul; the correlation coefficient between the observed and forecasted PM_2.5 concentrations was 0.78; the normalized mean error was 25%; the probability of detection for the events exceeding the national PM_2.5 standard was 0.81 whereas the false alarm rate was only 0.38. Both the hybrid bias correction technique and the Kalman filter bias adjustment technique were implemented into the GMAF as postprocessors. For the continuous and the categorical performance metrics examined, the Kalman filter bias adjustment technique performed better than the hybrid bias correction technique.

1. Introduction

Chemical transport models (CTMs) were first developed in the 1980s to investigate the transport [1,2,3] and the transformation [4,5] of air pollutants. The CTMs have been further developed by sophisticating atmospheric processes such as gas and aqueous chemical reactions [6], dry and wet deposition [7,8], and aerosol formation [9] to address mesoscale air pollution problems including acid deposition, tropospheric ozone (O₃), fine particulate matter (PM) pollution. The CTMs addressing air quality problems are named as air quality models. Most of mesoscale air quality models do not simulate atmospheric dynamics internally, requiring a mesoscale atmospheric model to provide the meteorological variables.

On the other hand, global atmospheric chemistry models have progressed rapidly to forecast and analyze the air quality despite of its coarse grid resolution and simplification of physical and chemical processes [10]. One of advantages of using global atmospheric chemistry models is that it is easy to incorporate polar orbiting satellite observations into data assimilation. The spatial and temporal resolution of polar orbiting satellite is too coarse to be used for mesoscale data assimilation. The remedy for this problem is to constrain the mesoscale model outputs by the global model outputs by using a grid nudging based four-dimensional data assimilation (FDDA) technique [11]. The grid nudging based FDDA is often used in mesoscale atmospheric models to perform continuous, long-term simulation and to improve the accuracy [12]. However, it has never been applied to mesoscale air quality models.

This paper presents the development of a global to mesoscale air quality forecast and analysis system (GMAF). The GMAF uses the weather research and forecasting model (WRF) and the community multiscale air quality model (CMAQ) as a mesoscale atmospheric model and a mesoscale air quality model, respectively. The grid nudging based FDDA was applied not only to the WRF but also to the CMAQ for the first time to link the corresponding global model. Variational assimilation of surface chemical monitoring data was not used because Korea has only a few surface monitoring stations in the adjacent upwind region by being surrounded by the ocean or the land with few surface monitoring stations.

The Copernicus atmosphere monitoring service (CAMS) forecast and reanalysis dataset was used as a gridded global atmospheric chemical composition field. The European centre for medium-range weather forecasts (ECMWF) extended its integrated forecast system (IFS) to carry out data assimilation and modeling of aerosols and chemically reactive gases [13]. Four-dimensional variational assimilation of the polar-orbiting satellite data was implemented to constrain atmospheric composition calculation. The grid nudging based FDDA using the CAMS dataset makes the aerosol and reactive gas concentrations of the CMAQ consistent with the satellite observations without explicitly using the satellite data.

The CMAQ is one of the most utilized community air quality models to address multiscale, multipollutant, multiphase air pollutant behaviors. The CMAQ offers multiple numerical solvers and chemical/physical modules so that a user may build his own model tailored to a particular application. Yamaji et al. (2020) [14] built as many as 28 CMAQ variations by selecting CMAQ versions, gas and aerosol chemical mechanisms, and grid configurations, let alone emission and boundary conditions. A successful application of the CMAQ depends on proper selection and adjustment of modules and rigorous evaluation utilizing observation data [15,16]. In this paper, the CMAQ results were compared with the observations from the criteria air pollutant monitoring network and the PM supersites located in Korea [17] to critically evaluate the selected modules and to modify them if necessary.

Every air quality modeling system including the CMAQ is subject to significant model errors caused by incompletely understood phenomena, parameterization induced error and uncertain emission inventories [18]. Observation interpolation and representativeness errors may cause additional errors [19]. As a way to reduce the errors, bias adjustment techniques have been developed. There are two kinds of bias adjustment techniques, the hybrid bias correction technique [20,21] and the Kalman filter bias adjustment technique [22]. The hybrid bias correction technique algebraically manipulates the forecast and observation at the previous time to reduce the bias whereas the Kalman filter bias adjustment technique uses adaptive and recursive method to optimally adjust the bias using the forecast and observation at the previous time. Both of these bias adjustment techniques were incorporated into the GMAF and tested.

2. Description of the GMAF

The framework, the major components, and the methodology of the GMAF are introduced in this section. Modification and update of the CMAQ to enhance the forecast performance are also described.

2.1. GMAF Configuration and Input Data

As shown in Figure 1, the GMAF is configured with two nested domains. The outer domain consists of 178 × 128 horizontal grids with 27 km spacing that cover Korean peninsula, most of China, Japan and parts of Russia, while the inner nest consists of 67 × 82 horizontal grids with 9 km spacing that cover South Korea, parts of North Korea and parts of Japan. A hybrid, sigma-pressure vertical coordinate was used with 23 levels and the top level was located at 200 hPa.

As shown in Figure 2, the GMAF imported CMAQ [23] for air quality forecast and analysis. The CMAQ version 5.3.1 was used with the carbon bond version 6 mechanism [24] as a gas-phase chemical mechanism, which includes 77 chemical species and 218 chemical reactions. The aerosol dynamics were modeled by AERO7 module, which includes 16 inorganic aerosol species, 33 organic aerosol species, as described in the github of CMAQ website [25]. The CMAQ divides sea salt into individual inorganic aerosol species and includes them except the coarse mode cations. Similarly, the CMAQ divides dust into individual aerosol species except the unknown composition fraction. The CMAQ treats the coarse mode cations of sea salt and the unknown composition fraction of dust as lumped aerosol species. The below-cloud scavenging module and the secondary organic aerosol formation mechanism were modified, and the evaporation loss of nitrate was added as detailed in Section 2.4. The WRF-ARW model version 3.6 [26] was used to provide the values of meteorological variables to the CMAQ.

Global assimilated datasets were used for generation of initial and boundary conditions of the outer domain and for providing gridded data to the grid nudging based FDDA. The WRF used the 0.25° global data assimilation system (GDAS) dataset and the CMAQ used the 0.4° CAMS dataset. The CAMS dataset was produced by the ECMWF’s IFS using four-dimensional variational data assimilation of the polar orbiting satellite data. A hybrid, sigma-pressure vertical coordinate was used with 137 levels and the top level was extended to 0.01 hPa. The CAMS dataset provided the concentration of 14 gaseous species and seven aerosol species in 3 h intervals [27]. The gaseous species were nitric oxide (NO), nitrogen dioxide (NO₂), sulfur dioxide (SO₂), formaldehyde (HCHO), ozone (O₃), peroxyacetyl nitrates (PAN), hydrogen peroxide (H₂O₂), nitric acid (HNO₃), hydroxyl radical (OH), carbon monoxide (CO), methane, ethane, propane, and isoprene. The aerosol species were dust, sea salt, hydrophobic black carbon, hydrophilic black carbon, hydrophobic organic matter, hydrophilic organic matter, and sulfate. The dust and the sea salt were distributed into the three size bins: 0.030–0.55, 0.55–0.9 and 0.9–20 µm for the dust and 0.03–0.5, 0.5–5 and 5–20 µm for the sea salt.

The clean air policy support system (CAPSS) emission inventory [28] was used to generate the gridded anthropogenic emission of South Korea. The Korea–United States air quality study (KORUS-AQ) anthropogenic emission inventory [29] was used for the rest of the region. The biogenic emissions were generated by MEGAN version 2.1 [30]. The CAPSS emission inventory is a national emission inventory compiled and published annually using the estimated or directly monitored emissions of CO, NO₂, SO₂, total suspended particulate (TSP), PM₁₀, PM_2.5, black carbon (BC), volatile organic compound (VOC), and ammonia (NH₃) from the point, the area and the mobile sources. The KORUS-AQ emission inventory was constructed from the CREATE emissions [31] to support the KORUS-AQ field campaign carried out in the spring of 2016.

The bias adjustment technique was applied to remove the bias of PM_2.5 forecast using the previous days’ PM_2.5 observation and forecast. The GMAF retrieved the hourly PM_2.5 measurements from the criteria air pollutant monitoring network with a 15 min delay. The collected hourly measurements were averaged at the end of the day to calculate the daily mean PM_2.5 concentration for the use of bias adjustment.

The three air monitoring networks in Korea measure the PM_2.5 and related species. Those are the criteria air pollutant monitoring network, the PM_2.5 speciation monitoring network and the PM_2.5 supersites. The criteria air pollutant monitoring network, consisting of 264 urban sites, 19 rural sites, three background sites as of 2016, measures the criteria air pollutant concentration including PM_2.5 mass concentration. The PM_2.5 speciation monitoring network and the PM_2.5 supersites measure the PM compositions such as sulfate, nitrate, ammonium, organic carbon, elemental carbon, and mineral species. The PM_2.5 compositions measured at the Bulkwang PM supersite were used for verification of the GMAF. The PM_2.5 mass concentrations measured at the criteria air pollutant network were used to evaluate the GMAF using both the continuous forecast performance metrics and the categorical forecast performance metrics.

2.2. Forecasting Period and Area

The first national air quality standard for PM_2.5 was promulgated in Korea in 2015 to begin installation of a PM_2.5 monitoring instrument at the criteria air pollutant monitoring station. Most of the criteria air pollutant monitoring stations in the urban area were equipped with a PM_2.5 monitoring instrument by 2016. Furthermore, KORUS-AQ field campaign was conducted in the spring of 2016 to provide the measurements from aircraft, ground sites and ships. The KORUS-AQ emissions were also developed to support the campaign [32]. Taking into account availability of monitoring data and emission inventory, this study chose the period from December of 2015 to November of 2016 for verification and evaluation of the GMAF. Four major cities in Korea were selected as target areas. Those are Seoul, Busan, Gwangju, and Daegu as displayed in Figure 3.

The PM_2.5 pollution in Seoul and Busan expressed in Figure 4, the first and second largest city in Korea, was the worst in 2016 in the period from 2015 to 2019. As shown in

Table 1. Annual mean concentrations and the number of days exceeding 35 μg/m³ of PM_2.5 in Seoul and Busan.

	Parameter	Annual Mean Concentrations (μg/m3)		Number of Days Exceeding the PM2.5 Threshold
Year		Seoul	Busan	Seoul	Busan
2015		23	26	42	66
2016		26	27	75	74
2017		25	26	63	61
2018		23	23	61	61
2019		25	21	64	41

, the annual mean PM_2.5 concentration was the highest and the number of days exceeding the PM_2.5 standard of 35 μg/m³ was the largest in 2016. Among meteorological parameters, precipitation, wind velocity and wind direction affect the year-to-year variations of the PM_2.5 concentrations the most. The precipitation from June to September of 2016 was 52% less than the 30-year average in Seoul. This low precipitation reduced the wet scavenging to increase the PM_2.5 concentrations. The PM_2.5 concentrations from June to September of 2016 were higher than the other year averages of the same period by 33%. The number of days of exceeding the PM_2.5 standard of 35 μg/m³ from June to September of 2016 was larger than the other year averages of the same period by 258% in Seoul as shown in Figure 4.

Busan is located about 320 km southeast of Seoul. The observed precipitation from May to August in 2016 was comparable to the 30-year average in Busan, and therefore the precipitation is unlikely to contribute to the PM_2.5 increase. Instead, the stagnant wind conditions that occurred in November 2016 increased the monthly mean concentrations and the number of days of exceedances as shown in Figure 5.

2.3. Grid Nudging Based FDDA Method

Data assimilation combines model with observation to improve performance of the forecast and the reanalysis [33]. The grid nudging based FDDA adds a relaxation term that is supposed to force the observations to the model [34] as

$$\frac{\mathrm{d}{Y}_{model}}{\mathrm{dt}}=F\left(x,Y_{model}\right)+W\left(x\right)\left(Y_{\mathrm{obs}}-Y_{\mathrm{model}}\right)$$

(1)

where Y is the dependent variable, F is a discretized form of the governing equation, x is the independent spatial variable, and W is the nudging coefficient. The subscripts ‘obs’ and ‘model’ denote the observations and the model, respectively. Global assimilated datasets may be used instead of observations to combine the mesoscale model with the global model. The dependent variables, Y, are defined as

$$Y= \alpha \left({\mathrm{p}}_{\mathrm{t}}-{\mathrm{p}}_{\mathrm{s}}\right) \mathrm{for} \mathrm{the} \mathrm{WRF},$$

(2)

$$Y= \phi J_s \mathrm{for} \mathrm{the} \mathrm{CMAQ},$$

(3)

where $\alpha$ is the meteorological variable of the WRF, $\phi$ is the chemical species concentration, p_t is the air pressure at the top of the domain, p_s is the air pressure at the surface and $J_s$ is the vertical Jacobian of the terrain-influenced coordinates. The variables nudged for the WRF were U, V winds, temperature and water vapor mixing ratio. The water vapor mixing ratio in the PBL and the surface temperature were not nudged. The nudging coefficients used for the WRF in this study were set as recommended by Baker et al. (2009) [35] and listed for completeness in

Table 2. Nudged variable and nudging coefficient.

	Nudged Variable	Nudging Coefficient
		Outer Domain	Inner Nest
WRF	U and V winds	5.0 × 10−4	2.5 × 10−4
	Temperature	5.0 × 10−4	2.5 × 10−4
	Water vapor mixing ratio	1.0 × 10−4	1.0 × 10−4
CMAQ	SO2, CO, NO, NO2, isoprene, O3, dust, sea salt	3.0 × 10−4	0

.

The nudged variables of the CMAQ were selected to be consistent with CAMS dataset. The primary pollutants were selected to minimize the emission uncertainties in the outer domain. The O₃ was added to the FDDA nudged variable list because its satellite observation data which the CAMS based on were reasonably accurate. In addition, the grid based FDDA on O₃ enables incorporation of stratospheric intrusion into the mesoscale model domain. The grid spacing of the CAMS dataset is 0.4° (about 40 km in mid-latitudes) and the grid spacing of outer domain and inner nest of the GMAF are 27 and 9 km, respectively. Interpolation of the CAMS dataset to the inner nest may severely underestimate the peak values due to a large difference between the grid resolutions. Moreover, the emission uncertainties in the inner nest, Korea emission, were reasonably small. Therefore, the grid nudging based on FDDA was applied only to the outer domain. Table 2 presents the list of nudged chemical species with the corresponding nudging coefficients.

2.4. Modification of the CMAQ

The CMAQ was modified to enhance the model accuracy based on the recent findings deduced from model verification as described below.

2.4.1. Below-Cloud Scavenging

The CMAQ assumes that accumulation mode and coarse mode particles are completely absorbed into cloud and rainwater. This assumption is valid only for a cloud environment where water droplets are readily formed around particles due to high water vapor contents but not valid for a rain environment. It may lead to overestimation of below-cloud scavenging and subsequently to underestimation of particulate matter concentrations in the below-cloud region.

In order to remove the complete absorption assumption in a rain environment, the CMAQ was modified to calculate the below-cloud scavenging rate by

$$\frac{\partial \mathrm{c}\left(\mathrm{t}\right)}{\partial \mathrm{t}}=-\mathsf{\Lambda }\mathrm{c}\left(\mathrm{t}\right)$$

(4)

where c(t) is the particulate species concentration at time t and the Λ is the scavenging coefficient [36]. The below-cloud scavenging coefficient was calculated by semiempirical formula derived by Slinn (1983) [37] with the empirically estimated mean raindrop diameter [38]. The calculated below-cloud scavenging coefficients ranged 10⁻⁶~10⁻² s, consistent with the field measurements [39].

If the particles are completely absorbed into rainwater as the CMAQ assumes, then the below-cloud scavenging coefficient should be equal to 1, which is several orders of magnitude larger than the one estimated here. More importantly, it remains constant whereas it should increase with the precipitation intensity. As a result, the CMAQ seriously overestimates the below-cloud scavenging rate and subsequently underestimates PM_2.5 concentration especially when the precipitation intensity is low.

The WRF-CMAQ simulation was performed from 21 March to 27 April in 2016 to test the modified below-cloud scavenging module. Figure 6 compares the modeled and observed precipitations at Seoul. The modeled and observed surface PM_2.5 concentrations were also compared. The modeled precipitation agrees very well with the observation except 21 April. The dates of the modeled precipitation between 0.1 mm/day and 1.0 mm/day are 29, 30 March, and 3, 6, 8, 9, 10, 14, 19, 22 April. As also shown in Figure 6, the CMAQ without modification considerably underestimated the PM_2.5 concentrations on these dates of little precipitation. As noted earlier, this is because the CMAQ severely overestimated the below-cloud scavenging rates by assuming complete absorption of particles into rainwater. The CMAQ with the proposed modification reduced overestimation of the below-cloud scavenging rates, resulting that the modeled PM_2.5 concentration agreed well with the observation.

2.4.2. Secondary Organic Aerosol Formation

In the present study, the AERO7 module including 12 primary organic aerosols (POA) and 21 secondary organic aerosols (SOAs) were used to model SOA formation. Compared to the previous versions of aerosol module, the AERO7 module expanded the SOA formation mechanism by adding pcSOA, organic nitrates from monoterpene oxidation and SOA from glyoxal and methylglyoxal. The pcSOA, standing for the potential SOA from combustion emissions, is a surrogate species designed to compensate missing and underestimated mass from combustion.

The pcSOA production rate of the AERO7 module was optimized for the southern and northern California sites and was known to overestimate the SOA in the continental US application [40]. Figure 7 displays the model calculated SOA compositions by season in Seoul. Here, the SOA was divided into four groups, pcSOA, SOA from biogenic VOCs (SOA_BVOC), SOA from anthropogenic VOCs (SOA_AVOC), and SOA from glyoxal and methylglyoxal (SOA_gly) for convenience. The pcSOA was the most dominant SOA in the winter due to low temperatures whereas the SOA_BVOC was the most dominant SOA in the summer due to high emission fluxes of biogenic VOCs and fast photochemical production. These seasonal characteristics of SOA compositions appeared similarly in most urban areas in Korea.

The CMAQ with the AERO7 module considerably overestimated the PM_2.5 mass in the winter. As the pcSOA was a dominant species in the winter, the PM_2.5 overestimation might be lessened by adjusting the pcSOA concentration by multiplying the pcSOA weighting factor (PWF). As shown in Figure 8, the AERO7 module with the PWF of 1.0 overestimated the observed monthly mean PM_2.5 concentrations at Seoul by 12~49%. As the PWF decreased to 0.4 and to 0.0, the modeled PM_2.5 concentrations were getting smaller. The modeled monthly mean PM_2.5 concentrations best agreed with the observations when the PWF was set to 0.4.

In addition, we conducted statistical performance evaluation to search an optimum value of the PWF. As listed in

Table 3. Statistical analysis on the model performance in forecasting PM_2.5 concentration at four cities (Seoul, Busan, Gwangju, Daegu) in 2016 with the PWF changing from 0 to 1.

Region Name	Weighting Factor	Continuous Forecast			Categorical Forecast
		R	NMB	NME	POD	FAR	HSS
Seoul	0.0	0.76	4%	22%	0.68	0.30	0.61
	0.2	0.77	9%	23%	0.75	0.35	0.61
	0.4	0.78	13%	25%	0.81	0.38	0.61
	0.7	0.78	19%	28%	0.83	0.47	0.53
	1.0	0.78	26%	33%	0.91	0.52	0.49
Busan	0.0	0.79	−20%	26%	0.39	0.16	0.47
	0.2	0.81	−17%	24%	0.44	0.14	0.52
	0.4	0.81	−14%	22%	0.51	0.16	0.58
	0.7	0.82	−9%	21%	0.57	0.20	0.60
	1.0	0.83	−5%	21%	0.64	0.25	0.63
Gwangju	0.0	0.75	−2%	25%	0.49	0.26	0.54
	0.2	0.77	2%	25%	0.53	0.36	0.51
	0.4	0.78	6%	25%	0.66	0.38	0.58
	0.7	0.80	12%	26%	0.77	0.39	0.62
	1.0	0.81	18%	29%	0.89	0.44	0.62
Daegu	0.0	0.77	−7%	23%	0.43	0.19	0.51
	0.2	0.78	−4%	22%	0.48	0.24	0.53
	0.4	0.78	−1%	22%	0.60	0.30	0.59
	0.7	0.79	4%	22%	0.69	0.32	0.62
	1.0	0.79	9%	24%	0.74	0.41	0.58

, the model performances in PM_2.5 forecast at four major cities in Korea in 2016 were evaluated by correlation coefficient (R), normalized mean bias (NMB), normalized mean error (NME), probability of detection (POD), false alarm ratio (FAR), and Heidke skill score (HSS), the definition and the usage of which were presented in Section 2.6. As the PWF increased, the R increased indicating performance improvement. However, there was no clear relation between the PWF and the NME. The POD increased with the PWF indicating performance improvement whereas the FAR decreased with the PWF indicating performance decline.

The R, the NMB, the NME, the POD, and the FAR are simple statistical metrics evaluating a partial performance of the model and therefore they may not agree with each other as happened here. This study used the HSS to decide an optimum value of the PWF, because the HSS evaluates the overall performance. The HSS averaged over the four cities was the highest when the PWF was equal to 0.4 or 0.7. The HSS at Seoul, the most populated city in Korea, was the highest when PWF was equal to 0.4. This value of the PWF is the same as the value previously derived using Figure 8. Therefore, the PWF was set equal to 0.4 in this study.

The PM supersite located at Bulkwang in Seoul collected PM_2.5 on a quartz filter every three days and analyzed them for organic carbon (OC). The observed OC at the Bulkwang PM supersite was compared with the modeled OC in Figure 9. The CMAQ calculated organic matter (OM) concentrations instead of OC concentrations. Therefore, the calculated OM concentrations were converted to the OC concentrations by taking account of molecular formula of individual organic aerosol species constituting the OM. As shown in Figure 9, the modeled OC agreed well with the observation except the summer. As the pcSOA dominated except the summer, the model must have accurately estimated the pcSOA concentration with the PWF of 0.4.

One other thing to note is that the model significantly over-predicted the OC in the summer. As most of SOA in this period is the SOA from biogenic VOCs as shown in Figure 7, it is highly likely that the emissions of biogenic VOCs, isoprene and terpenes were overestimated or that the conversion rates from the biogenic VOCs to the SOA were overestimated. More studies are necessary to address this over-prediction problem.

2.4.3. Evaporation Loss of Nitrate

A national reference method for PM_2.5 in many countries including Korea specifies Teflon membrane filter for collection of ambient particles. Teflon membrane filter may cause a significant evaporation loss of particulate nitrate (NO₃⁻) during and after the sampling in the warm environment [41,42]. Assuming equilibrium, the evaporation loss ($\Delta {\mathrm{NO}}_3^{-})$ is expressed as

$$\Delta {\mathrm{NO}}_3^{-}=\frac{745.7}{{\mathrm{T}}_{\mathrm{R}}}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=24}\frac{\sqrt{{\mathrm{K}}_{\mathrm{i}}}}{24},$$

(5)

where T_R is the daily average temperature of the sampled air and K_i is the dissociation constant for ammonium nitrate evaluated at the ambient temperature and the relative humidity for hour i as described by Frank (2006) [43].

The GMAF applied Equation (5) to the temperature and humidity calculated by the WRF to estimate the nitrate evaporation loss. The estimated nitrate evaporation loss was subtracted from the nitrate concentration calculated by the CMAQ to estimate the nitrate concentration left on the filter after evaporation.

2.5. Implementation of Bias Adjustment Techniques

The accuracy of air quality forecast may be improved by applying bias adjustment techniques. Both the hybrid bias correction technique and the Kalman filter bias adjustment technique were implemented into the GMAF as postprocessors. The hybrid bias correction technique uses either the difference or the ratio between the forecast and the observation [44]. The former is an additive correction and expressed as

$${\mathrm{C}}_{\mathrm{t}+\Delta \mathrm{t}}={ \mathrm{F}}_{\mathrm{t}+\Delta \mathrm{t}}+\frac{1}{\mathrm{Nday}} {{\displaystyle \sum }}_{\mathrm{j}=\mathrm{Nday}-1}^{\mathrm{j}=0}({\mathrm{O}}_{\mathrm{t}-\mathrm{j}\Delta \mathrm{t}}-{\mathrm{F}}_{\mathrm{t}-\mathrm{j}\Delta \mathrm{t}}).$$

(6)

The latter is a multiplicative correction and expressed as

$${\mathrm{C}}_{\mathrm{t}+\Delta \mathrm{t}}={ \mathrm{F}}_{\mathrm{t}+\Delta \mathrm{t}}\times \frac{{{\displaystyle \sum }}_{\mathrm{j}=\mathrm{Nday}-1}^{\mathrm{j}=0}{\mathrm{O}}_{\mathrm{t}-\mathrm{j}\Delta \mathrm{t}}}{{{\displaystyle \sum }}_{\mathrm{j}=\mathrm{Nday}-1}^{\mathrm{j}=0}{\mathrm{F}}_{\mathrm{t}-\mathrm{j}\Delta \mathrm{t}}},$$

(7)

where C is the corrected forecast value, F is the raw forecast value and O is the observation. Nday denotes the number of previous days representing a training period. Typical values for Nday are the numbers between 1 and 5.

Kalman filter bias adjustment technique predicts the future bias by correcting the previous estimate of the bias using the difference between the bias estimate and the forecast error at the previous time. The predicted future bias is used to adjust the forecast. The Kalman filter bias adjustment technique was implemented into the GMAF by following the work by Delle Monache et al. (2006) [22]. Kalman filter bias adjustment technique is sensitive to the error ratio, which determines the relative weighting between the observation and the forecast and needs to be optimized.

2.6. Forecast Performance Evaluation Metrics

The methods for forecast performance evaluation are introduced in this section. Selected forecast period and variables for forecast evaluation are also discussed.

2.6.1. Forecast Variables

The GMAF can forecast most of the criteria air pollutants but this study selected the PM_2.5 for performance analysis. As of 2016, 25 PM_2.5 monitoring stations were located in Seoul, 19 stations in Busan, six stations in Gwangju and seven stations in Daegu. Korea uses the PM_2.5 concentration averaged over all the stations in the city of interests to set a policy goal and to calculate the air quality index (AQI). Taking averages over the city improves the spatial representativeness of monitoring data and reduces the associated errors.

Although the recent advances in analytical chemistry have made it possible to measure PM_2.5 mass and composition at time intervals equal to or less than 1 h, most of PM_2.5 toxicity and PM_2.5 management studies still use the daily mean or annual mean concentrations as a metric of PM_2.5. This study used the daily mean PM_2.5 concentration for forecast performance evaluation.

2.6.2. Performance Evaluation Metrics of Continuous Forecasts

Continuous forecast aims at forecasting the values of continuous variables. Temperatures, wind velocities and air pollutant concentration are good examples of continuous variables. The model performance of continuous forecast was measured by three statistical metrics in this study; R, NMB and NME. The R, correlation coefficient, gives a measure of linear relationship between two variables and ranges between −1 and 1. The R between the forecast and observed values is defined as

$$\mathrm{R}=\frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{N}}\left({\mathrm{O}}_{\mathrm{i}}-\overline{\mathrm{O}}\right)\left({\mathrm{F}}_{\mathrm{i}}-\overline{\mathrm{F}}\right)}{\sqrt{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{N}}{\left({\mathrm{O}}_{\mathrm{i}}-\overline{\mathrm{O}}\right)}^2}\sqrt{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{N}}{\left({\mathrm{F}}_{\mathrm{i}}-\overline{\mathrm{F}}\right)}^2}},$$

(8)

where O_i and F_i are the observation and the forecast, respectively, and N is the number of paired data sets.

The NMB, normalized mean bias, is the difference between observed and forecasted mean value normalized by the observed mean value and is defined by

$$\mathrm{NMB}=\frac{\overline{\mathrm{F}}-\overline{\mathrm{O}}}{\overline{\mathrm{O}}}\times 100.$$

(9)

The NMB is a useful metric for measuring the systematic errors. However, as the number of datasets increases, under-forecasted values tend to offset over-forecasted values to improve the NMB even if the accuracy remains unchanged. The root mean square (RMSE) uses the square of the difference between the observed and forecasted value to measure the errors as defined by

$$\mathrm{RMSE}=\sqrt{\frac{1}{\mathrm{N}}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{N}}{({\mathrm{F}}_{\mathrm{i}}-{\mathrm{O}}_{\mathrm{i}})}^2}.$$

(10)

The squaring operation makes the RMSE strongly dominated by the largest errors. The RMSE is not recommended for the air quality forecast and analysis because of its high sensitivity to the largest concentrations [10,45].

The normalized mean error (NME) uses the absolute value of the difference instead of the squares of them as defined by

$$\mathrm{NME} =\frac{1}{\mathrm{N}}\frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{i}=\mathrm{N}}\left|{\mathrm{F}}_{\mathrm{i}}-{\mathrm{O}}_{\mathrm{i}}\right|}{\overline{\mathrm{O}}}\times 100.$$

(11)

The NME gives a measure of the overall forecast error and is used in this study instead of RMSE.

Emery et al. (2017) [45] analyzed the model performance evaluation studies published from 2006 to 2012 in the United States and recommended the criteria and goal of the R, the NMB and the NME as listed in

Table 4. Goals and criteria of the R, the normalized mean bias (NMB) and the normalized mean error (NME) for the forecast of daily mean PM_2.5 concentrations.

R		NMB		NME
Goal	Criteria	Goal	Criteria	Goal	Criteria
>0.70	>0.40	<±10%	<±30%	<±35%	<±50%
R = 1 is perfect correlation R = 0 is no correlation		NMB < 0 is under-forecast NMB > 0 is over-forecast		NME = 0 is perfect forecast

, which is used in this study as a reference.

2.6.3. Performance Evaluation Metrics of Categorical Forecasts

Categorical forecast aims at forecasting the occurrence of a specific event. The event of interests in this study is that the PM_2.5 concentration exceeds the Korean PM_2.5 standard of 35 μg/m³. The contingency table used for analysis of the relationship between two categorical variables is presented in

Table 5. Contingency table structure.

Event Forecast	Event Observed
	Yes	No
Yes	A	B
No	C	D

, where the “A” is the number of event forecasts with event occurring. The “B” is the number of event forecasts with event not occurring. The “C” is the number of no-event forecasts with event occurring. Finally, the “D” is the number of no-event forecasts with event not occurring.

The A, B, C, D in the contingency table may formulate various performance metrics, including probability of detection (POD), probability of false detection (PODF), false alarm ratio (FAR), success ratio (SR), critical success index (CSI), true skill statistic (TSS), Gilbert skill score (GS), Heidke skill score (HSS), bias (B), and accuracy (A) [46].

The POD ranges from 0 to 1 with 1 being a perfect forecast and is defined by

$$\mathrm{POD}= \frac{\mathrm{A}}{\left(\mathrm{A}+\mathrm{C}\right)}.$$

(12)

The POD considers missed events only, neglecting false alarms and therefore it can be artificially improved by over-forecasting. The FAR complements the POD in this aspect and is defined by

$$\mathrm{FAR}= \frac{\mathrm{B}}{\left(\mathrm{A}+\mathrm{B}\right)}.$$

(13)

The FAR ranges from 0 to 1 with 0 being a perfect forecast. The FAR considers false alarm events only, neglecting missed events and therefore it can be artificially improved by under-forecasting.

The POD and the FAR are examples of simple skill metrics. The problem of simple skill metrics is that a good score does not necessarily mean a good overall forecasting performance. One can overcome this problem by combining two or more simple skill metrics. This kind of skill metrics, called here as complex skill metrics, are the CSI, the TSS and the HSS. The CSI and the TSS may not work properly for rarely occurring events. As exceedance of the PM_2.5 standard, the event in this study, may occur rarely in certain seasons and regions, we selected the HSS as a complex skill metric.

The HSS is the fraction of accurate forecast with correction of random forecasts as defined by

$$\mathrm{HSS}= \frac{\mathrm{A}+\mathrm{D}-\mathrm{E}}{\mathrm{A}+\mathrm{B}+\mathrm{C}+\mathrm{D}-\mathrm{E}},$$

(14)

where E is the number of correct random forecasts as defined by

$$\mathrm{E}= \frac{\left(\mathrm{A}+\mathrm{B}\right)\left(\mathrm{A}+\mathrm{C}\right)+\left(\mathrm{B}+\mathrm{D}\right)\left(\mathrm{C}+\mathrm{D}\right)}{\mathrm{A}+\mathrm{B}+\mathrm{C}+\mathrm{D}}.$$

(15)

The HSS ranges from −∞ to 1 with 1 being a perfect forecast. The HSS above zero means that the forecast is better than a random guess and the HSS above 0.5 is deemed a good score.

Studies on categorical forecast performance evaluation of PM_2.5 forecast were much more limited than studies on continuous forecast performance evaluation [47]. The POD for the event of exceeding the 8-h O₃ standard in the Northeastern United States ranged between 0.07 and 0.37 whereas the FAR, often larger than the POD, ranged between 0.64 and 0.76 [48]. The POD for the event of exceeding the 24-h PM_2.5 standard in the Eastern United States was 0.28 whereas the FAR was 0.35 [49]. The POD for the event of exceeding the 24-h PM_2.5 standard in Korea ranged between 0.1 and 0.39 whereas the FAR ranged between 0 and 0.64 [50].

3. Results and Discussion

The performance evaluation results of the GMAF are presented in this section. Test results of performance improvement attained by incorporating the nitrate evaporation and the bias adjustment techniques are also presented.

3.1. Performance Evaluation of the Base Forecast

The GMAF was used to forecast the PM_2.5 for the year 2016. All the modifications described in Section 2.4 were included except nitrate evaporation loss.

Table 6. Performance evaluation of continuous and categorical forecast at Seoul, Busan, Gwangju, Daegu in 2016. A threshold value for the categorical forecast was set to the daily mean PM_2.5 35 μg/m³.

		PM2.5,obs #	Continuous Forecast			N_Day %	Categorical Forecast
			R	NMB	NME		POD	FAR	HSS
Seoul	Winter	26.4 μg/m3	0.87	26%	27%	17	1.00	0.48	0.58
	Spring	30.2 μg/m3	0.78	3%	21%	31	0.90	0.15	0.81
	Summer	23.8 μg/m3	0.72	12%	26%	12	0.58	0.53	0.44
	Fall	23.8 μg/m3	0.77	12%	25%	15	0.60	0.50	0.45
	Year	26.0 μg/m3	0.78	13%	25%	75	0.81	0.38	0.61
Busan	Winter	30.2 μg/m3	0.86	−16%	20%	22	0.68	0.17	0.68
	Spring	30.8 μg/m3	0.72	−11%	23%	25	0.52	0.24	0.51
	Summer	21.8 μg/m3	0.81	−9%	20%	9	0.22	0.00	0.34
	Fall	24.5 μg/m3	0.79	−19%	25%	14	0.43	0.00	0.56
	Year	26.8 μg/m3	0.81	−14%	22%	70	0.51	0.16	0.58
Gwangju	Winter	25.9 μg/m3	0.83	11%	24%	22	0.86	0.27	0.72
	Spring	28.0 μg/m3	0.70	−1%	25%	21	0.52	0.42	0.43
	Summer	18.6 μg/m3	0.68	11%	27%	3	0.00	1.00	−0.03
	Fall	21.3 μg/m3	0.80	7%	23%	7	0.71	0.44	0.59
	Year	23.5 μg/m3	0.78	6%	25%	53	0.66	0.38	0.58
Daegu	Winter	28.6 μg/m3	0.82	−6%	19%	25	0.72	0.18	0.68
	Spring	26.3 μg/m3	0.77	2%	21%	16	0.69	0.35	0.59
	Summer	20.0 μg/m3	0.64	7%	27%	2	0.00	1.00	−0.03
	Fall	23.4 μg/m3	0.79	−5%	21%	15	0.40	0.25	0.46
	Year	24.6 μg/m3	0.78	−1%	22%	58	0.60	0.30	0.59

^# observed seasonal or yearly averaged concentration of PM_2.5. ^% number of days exceeding PM_2.5 standard of 35 μg/m³.

shows the seasonal and yearly values of performance metrics, where the seasons were defined as winter (December, January, February), spring (March, April, May), summer (June, July, August), and fall (September, October, November). The average PM_2.5 concentrations in the winter and the spring were higher than those in the summer and the fall in all the four selected cities, which are very typical of the PM_2.5 concentrations in Korea. The annual values of R were higher than 0.78 in all four cities, satisfying the goal set in Table 4. The winter values of R were the higher than the other seasons and the summer values of R were lower than the other seasons except Busan.

The NMBs at Seoul were positive in all the seasons, indicating over-forecasting, whereas those at Busan were all negative, indicating under-forecasting. Out of total 16 cases (four seasons and four cities), half of the cases had the NMB value less than ±10%, achieving the goal set in Table 4 and the other half of cases had the NMB value less than ±30%, satisfying the criteria also set in Table 4. The seasonal NME ranged between 19% and 27%, achieving the goal set in Table 4. Unlikely to the R, the NME did not exhibit a distinctive seasonal variation.

The number of days exceeding the 24-h PM_2.5 standard, which is the frequency of occurrence of events, ranged between 17 and 31 days during the winter and the spring because of severe PM_2.5 pollution in Korea. The frequency of occurrence of events in this study is several times higher than that in the literature quoted in Section 2.6.3. It may be noted here that categorical forecasts perform better for the frequent events than for the rare events. During the winter, the skill scores were very good; the POD were 0.68~1.0 and exceeded the FAR by 0.41~0.59, and HSS were 0.58~0.68. Although the skill scores went down in the spring and fall, all three skill scores were still deemed fairly good.

The categorical forecast performed poorly during the summer compared to the other seasons because of two reasons. Firstly, as noted above, the R values and the NME values in the summer were generally lower than those in the other seasons, indicating the GMAF might not perform well in the summer. Secondly, the events occurred less frequently in the summer due to lessened PM_2.5 pollution in Korea, which might worsen the categorical skill scores measured by the POD, the FAR and the HSS.

As discussed above, both the GMAF performed the best during the winter and worst during the summer. In order to investigate the effects of seasons on the forecast performance, the modeled seasonal PM_2.5 chemical compositions are displayed in Figure 10. The nitrate was a dominant PM_2.5 species in the winter. The PM_2.5 nitrate in the urban area is rather locally produced than regionally transported [51]. The local emissions of nitrogen oxides (NO_x) were well known and the homogeneous and heterogeneous chemical pathways were well defined such that the GMAF was able to accurately predict the PM_2.5 nitrate concentrations and subsequently the PM_2.5 mass concentrations during the winter.

During the summer, the PM_2.5 mass was dominated by the OM, the most of which was the SOA from biogenic VOC as shown in Figure 7. There was large uncertainty in estimating biogenic VOC emission and in modeling aerosol formation processes from biogenic VOCs, which probably degraded the model performance.

3.2. Modeling of Nitrate Evaporation Loss

The PM supersite located at the Bulkwang in Seoul measured the PM_2.5 mass and PM_2.5 nitrate using Teflon membrane filter. Teflon membrane filter causes a significant evaporation loss of particulate nitrate during and after the sampling in the warm environment. Here, the nitrate before evaporation loss is defined as as-is nitrate, whereas the nitrate left on the filter after evaporation loss is defined as retained nitrate.

The monthly mean concentrations of the retained nitrate at the Bulkwang site were calculated by applying Equation (5) to the temperature and the humidity forecasted by WRF and the nitrate concentration modeled by CMAQ. As displayed in Figure 11, both the as-is and retained PM_2.5 nitrate concentration were the highest during the winter months and the lowest during the summer months because the ratios of nitrate aerosol to nitric acid gas (NO₃⁻/HNO₃) decrease with temperature increase. In the summer months, the nitric acid mainly stayed in the gas phase due to the warm temperature. It may be noted that the nitric acid gas is readily removed by wet and dry deposition to have shorter lifespan than the nitrate aerosol.

The difference between as-is nitrate and retained nitrate is the evaporation loss of nitrate taking place during and after Teflon filter sampling. The evaporation loss is less than 10% of as-is nitrate during the winter months whereas it was more than 80% of as-is nitrate during the summer months. Therefore, the retained nitrate was negligibly small by being less than 0.5 μg/m³ during the summer months.

Figure 12 compares the observed nitrate with the modeled nitrate between April and October during which Equation (5) predicted a significant evaporation loss of nitrate as shown in Figure 11. As the nitrate was measured every three days, the modeled nitrate concentrations were also selected every three days in the Figure 12. The modeled retained nitrate agreed well with the observed nitrate in April, August, September, and October, indicating that Equation (5) calculated the nitrate evaporation loss accurately. The retained nitrate concentrations were kept below 5 μg/m³ to lessen the nitrate overestimation appeared in the as-is nitrate concentration. However, the observed nitrate was closer to the modeled as-is nitrate than the modeled retained nitrate about half of May, June and July. This underestimation of the retained nitrate is probably due to inaccurate estimation of the temperature to which the nitrate evaporation loss is sensitive.

Table 7. Effect of inclusion of nitrate evaporation loss on the performance of continuous and categorical forecast at Seoul, Busan, Gwangju, Daegu during spring and fall in 2016. A threshold value for the categorical forecast was set to the daily mean PM_2.5 35 μg/m³.

		Nitrate Evaporation Loss	Continuous Forecasting			Categorical Forecasting
			R	NMB	NME	POD	FAR	HSS
Seoul	Spring	Not included	0.78	3%	21%	0.90	0.15	0.81
		Included	0.76	0%	22%	0.68	0.13	0.67
	Fall	Not included	0.77	12%	25%	0.60	0.50	0.45
		Included	0.77	4%	23%	0.33	0.50	0.31
Busan	Spring	Not included	0.72	−11%	23%	0.52	0.24	0.51
		Included	0.67	−13%	26%	0.52	0.24	0.51
	Fall	Not included	0.79	−19%	25%	0.43	0	0.56
		Included	0.78	−22%	27%	0.5	0	0.63
Gwangju	Spring	Not included	0.70	−1%	25%	0.52	0.42	0.43
		Included	0.64	−6%	27%	0.33	0.42	0.31
	Fall	Not included	0.80	7%	23%	0.71	0.44	0.59
		Included	0.81	−2%	22%	0.29	0.33	0.37
Daegu	Spring	Not included	0.77	2%	21%	0.69	0.35	0.59
		Included	0.72	−3%	23%	0.56	0.25	0.58
	Fall	Not included	0.79	−5%	21%	0.4	0.25	0.46
		Included	0.80	−10%	22%	0.4	0	0.53

shows the effect of inclusion of nitrate evaporation loss on the forecast performance in the spring and the fall during which the nitrate evaporation loss had a discernible effect on the PM_2.5 mass concentration. The CTMs including the CMAQ are exposed to high degrees of uncertainty in modeling physical and chemical processes and in preparing input data such as emissions, meteorological parameters. Enhancement of one module does not necessarily mean improvement of the forecast performance. As the nitrate evaporation loss was included, the NMB decreased as expected. The R decreased in all the four cities for the spring whereas it increased in Gwangju and Daegu for the fall. The NME increased in most cases. Therefore, it may be concluded that the performance of the continuous forecast was not improved although the nitrate evaporation loss was included. Moreover, the categorical forecast measured by the POD, the FAR and HSS was declined in most cases considered here.

3.3. Application of Bias Adjustment Technique

The bias adjustment techniques described in Section 2.5 were applied to the PM_2.5 forecasts at the four cities for the year 2016. We tested different values of the number of previous days (Nday) for additive and multiplicative bias correction and different values of the error ratio for the Kalman filter bias adjustment, and chose 2 as the number of previous days and 0.5 for the error ratio.

The additive and multiplicative bias correction techniques were designed to remove the biases by algebraic manipulation. They successfully eliminated the biases at all the four cities as shown in

Table 8. Performance comparison of continuous and categorical forecast between the additive bias correction, multiplicative bias correction, Kalman filter bias adjustment at Seoul, Busan, Gwangju, Daegu in 2016.

		Continuous Forecast			Categorical Forecast
		R	NMB	NME	POD	FAR	HSS
Seoul	Raw forecast	0.78	13%	25%	0.81	0.38	0.61
	Additive bias correction	0.78	0%	23%	0.68	0.32	0.59
	Multiplicative bias correction	0.80	1%	21%	0.65	0.34	0.57
	Kalman filter bias adjustment	0.82	7%	21%	0.80	0.37	0.62
Busan	Raw forecast	0.81	−14%	22%	0.51	0.16	0.58
	Additive bias correction	0.78	0%	21%	0.65	0.32	0.58
	Multiplicative bias correction	0.76	1%	23%	0.68	0.33	0.59
	Kalman filter bias adjustment	0.83	−7%	19%	0.62	0.24	0.62
Gwangju	Raw forecast	0.78	6%	25%	0.66	0.38	0.58
	Additive bias correction	0.79	0%	25%	0.68	0.29	0.64
	Multiplicative bias correction	0.78	1%	25%	0.62	0.39	0.55
	Kalman filter bias adjustment	0.83	3%	22%	0.75	0.25	0.71
Daegu	Raw forecast	0.78	−1%	22%	0.60	0.30	0.59
	Additive bias correction	0.77	0%	23%	0.67	0.34	0.59
	Multiplicative bias correction	0.75	1%	23%	0.67	0.38	0.57
	Kalman filter bias adjustment	0.82	−1%	20%	0.70	0.26	0.66

. However, the improvements of biases did not propagate to the other performance evaluation metrics. They did not generally improve the R and the NME except for Seoul. Moreover, they generally degraded the POD and the HSS.

Although the Kalman filter bias adjustment technique was less efficient in reducing the NMB than the additive and multiplicative bias correction techniques, it improved all the other performance metrics significantly. The averaged values over the four cities were improved by 5% for the R, 13% for the NME, 11% for the POD, 8% for the FAR, and 10% for the HSS.

4. Conclusions

The GMAF was developed by utilizing the currently available multiscale air quality modeling system and the global forecast and analysis dataset. The GMAF implemented the grid nudging based FDDA onto the CMAQ for the first time to take advantage of data assimilation done in the global modeling. Although the current forecast verification was limited to the next day forecast and analysis, the forecasting period can be extended up to five days permitted by the global forecast and analysis model without a significant loss of accuracies.

The GMAF lessened the tendency of underestimating the PM_2.5 concentrations in the case of little precipitation by improving the below-cloud scavenging coefficient estimation. The CMAQ-5.3.1 with carbon bond version 6 mechanism and AERO7 module significantly enhanced the accuracy of nitrate formation rate calculation by updating homogenous reaction rate constants and heterogeneous reaction mechanism. The SOA formation mechanism was also expanded by adding pcSOA, organic nitrates from monoterpene oxidation and SOA from glyoxal and methylglyoxal. The expansion of the SOA formation mechanism amended the underestimation problem of SOA frequently found in the previous version. However, a comparison with the observations indicated that the pcSOA in the winter and the SOA from biogenic VOCs in the summer might be overestimated. A more refined estimation of the pcSOA emissions and more accurate modeling of biogenic VOCs are recommended as future studies.

A severe PM_2.5 episode often arises in the winter and the spring in Korea. The Korea government agency relies on the numerical forecast during these PM_2.5 season in issuing a PM_2.5 advisory or warning to the people. The GMAF performed very well especially in the winter suiting the needs of the government. In the winter, the R between the observed and forecasted PM_2.5 was 0.87, the POD for event exceeding the national PM_2.5 standard was 1.00, and the FAR was only 0.48 at Seoul. The GMAF also showed good performance at the other three cities, Busan, Gwangju and Daegu.