Next Article in Journal
The Effects of Nitrogen Fertilisation on Plant Species Richness in European Permanent Grasslands: A Systematic Review and Meta-Analysis
Previous Article in Journal
Soil Carbon and Nitrogen Stocks under Agrosilvopastoral Systems with Different Arrangements in a Transition Area between Cerrado and Caatinga Biomes in Brazil
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Improving Simulations of Rice in Response to Temperature and CO2

1
Adaptive Cropping Systems Laboratory, USDA-ARS, Beltsville, MD 20705, USA
2
Texas A&M AgriLife Research Center at Beaumont, 1509 Aggie Drive, Beaumont, TX 77713, USA
3
Floral and Nursery Plants Research, USDA-ARS National Arboretum, Beltsville, MD 20705, USA
4
Nebraska Water Center, University of Nebraska Lincoln, Lincoln, NE 68588, USA
5
Department of Plant Science and Landscape Architecture, University of Maryland, College Park, MD 20742, USA
*
Author to whom correspondence should be addressed.
Agronomy 2022, 12(12), 2927; https://doi.org/10.3390/agronomy12122927
Submission received: 23 September 2022 / Revised: 18 November 2022 / Accepted: 21 November 2022 / Published: 23 November 2022

Abstract

:
Crop models are frequently used to assess the impact of climate change responses. Evaluation of model performance against empirical data is crucial to establish confidence, particularly for rice (Oryza sativa L.), one of the world’s important cereal crops. Data from soil-plant-atmosphere-research (SPAR) chambers and field plots were used to assess three versions of the ORYZA model to a range of climate conditions. The three versions were: V1–the original, V2–V1 plus a revised heat stress component, and V3–V2 plus a coupled leaf-level gas exchange algorithm. Comparison against SPAR datasets, which covered a range of temperatures at two CO2 levels, indicated successive improvement in yield predictions with the model version. Root Mean Square Error (RMSE) decreased by 520 and 647 kg ha−1 for V2 and V3, respectively, and Wilmott’s index of agreement improved by 10 and 12% compared with V1 when averaged across 20 treatments and three cultivars. Similar improvements were observed from 17 field dataset simulations with two additional varieties. These results indicated the importance of improving heat sterility functions and carbon assimilation methodologies that incorporate direct responses to air temperature and CO2 concentration in rice models. Accounting for cultivar differences in thermal sensitivity is also an important consideration for climate assessments.

1. Introduction

Rice (Oryza sativa L.) is the main staple food for over half the world’s population and is of particular importance to global food security [1,2]. The crop is vulnerable to high air temperatures, and projected climate changes are thus of great concern [3,4] given that a substantial portion of current production regions are already at, or above, thresholds associated with heat stress. While rice shows positive yield and growth responses to elevated atmospheric carbon dioxide concentration (CO2) [5,6,7], there is evidence that the CO2 fertilization effect is minimized as temperatures increase [3,8]. Explanatory crop models are frequently used to assess and identify the impact and potential adaptation strategies associated with current and future climate impacts on crop production [9,10,11,12,13]. However, uncertainties remain regarding the ability of rice crop models to simulate yield responses to rising CO2 and temperatures [14,15,16]. The current study focuses on the extent to which changes to thermal stress and representation of gas exchange processes in a popular rice model resulted in improved yield and biomass predictions.
Implementation of heat responses on plant growth and development, particularly at high and low temperature extremes, is one of the largest sources of uncertainty among crop models [15,17]. This is a particular concern for rice, where even a few hours of high-temperature exposure (e.g., above 33.7 °C) can reduce pollen viability and spikelet fertility, with a consequent reduction in grain yield [18,19]. Popular rice models such as ORYZA [20,21] incorporate the effects of a broad temperature range on developmental rates, leaf expansion, photosynthesis and respiration rates, and other growth and development components. However, predictions of high or low-temperature impacts on yield can be inaccurate [22,23] despite options for simulating high heat stress and cold sterility effects on spikelet fertility and leaf expansion rate. Similarly, a recent rice model intercomparison study [24] found that 14 crop models underestimated the negative effects of heat stress on grain yield due to a lack of full accounting for short-term extreme heat exposure during anthesis.
Simulated responses to elevated CO2 are generally more consistent among crop models than temperature extremes [25], but methodologies vary considerably. This can lead to differences when projecting interactions with temperature and other factors. A recent survey of crop modeling approaches indicated that empirical adjustments to radiation use efficiency (RUE) and/or transpiration efficiency coefficients are the most frequently used [10]. While fairly robust, these and other empirical approaches can result in inaccuracies when assessing the effects of CO2 along with high growth temperatures and limited water availability [26]. Predicting the response of rising CO2 and temperature is complicated by the influence CO2 indirectly exerts on leaf temperature due to stomatal closure and evaporative cooling [27,28]. Mimicking the photosynthetic acclimation effect [26] in models is also challenging. Methods that directly account for the influence of elevated CO2 on stomatal closure and link gas exchange with an energy balance at leaf surfaces have therefore been suggested as more appropriate for climate change studies [29]. Coupled gas exchange–energy balance methodologies have been recently implemented and tested in corn, potato, and soybean models [30,31,32]. A related energy balance approach using sub-models for photosynthesis, stomatal conductance, and transpiration was recently incorporated into ORYZA [33] and shown to provide more accurate estimates of canopy photosynthesis under different CO2 levels over the growing season as compared with the original model. However, this modification has not been fully evaluated, particularly in conjunction with varying growth temperature levels.
The objective of this study was to evaluate the effectiveness of gas exchange, and heat stress modifications implemented in the ORYZA rice model with respect to five cultivars. In this effort, we (1) initially modified the model to more accurately simulate growth duration over the broader temperature range common to U.S. conditions [23], (2) modified and parameterized the methods for simulating the impact of high heat on spikelet sterility, and (3) tested the extent to which adding the heat stress routine along with an existing coupled leaf gas exchange–energy balance model would improve predictive capability in reproducing effects of temperature and CO2 on yield and above-ground biomass. Results highlight the effect these incremental improvements have on increasing the accuracy of rice model predictions for current and possible future climate impacts, which can serve as foundational pieces for improvements to other crop models.

2. Materials and Methods

2.1. Datasets

Soil-Plant-Atmosphere-Research (SPAR) growth chamber data were used to evaluate different model version responses for three cultivars grown across a wide range of growth temperatures and two CO2 levels. Field data from the U.S. Mississippi Delta were used to further test the response of two additional cultivars over several years of data with different planting dates. Above-ground biomass and grain yield were obtained from multiple experiments previously conducted in SPAR chambers located at the United States Department of Agriculture—Agricultural Research Service (USDA-ARS), Beltsville, Maryland, and the University of Florida, Gainesville, Florida, U.S. The experimental data were reported in [6,34,35,36,37,38] and utilized IR30, Cocodrie, and Jefferson cultivars (Table S1). The IR30 cultivar studies included 17 datasets. Five temperature treatments ranged from 25/18 °C to 37/30 °C day/night thermoperiods under elevated CO2 (660 ppm). Two additional treatments at 28/21 °C and 31/31 °C were conducted at both ambient and elevated CO2 levels, along with an additional study at 31/31 °C with six different CO2 levels. Data for Cocodrie and Jefferson cultivars came from [6] and included four constant 24 h air temperature treatments of 24, 28, 32, and 36 °C at elevated CO2 (700 ppm) with an additional temperature treatment of 28 °C at ambient CO2. Air temperature, CO2 concentration, and active photosynthetic radiation (PAR) were available on-site for Jefferson and Cocodrie cultivars. Solar radiation values were obtained at the Florida location from AgMERRA and AgCFSR datasets [39].
Experimental field data were obtained from performance trials of paddy-grown rice within the U.S. Mississippi Delta [40]. Data for two varieties, a heat stress tolerant hybrid, Clearfield CLXL753 (RiceTec, Inc., Alvin, TX, USA; https://www.ricetec.com/wp-content/uploads/2017/10/2018-Product-Characteristics_HR.pdf accessed on 20 November 2022), and a conventional tropical japonica variety, Wells, were selected based on prominence in the dataset. The data ranged from 2012 to 2015 and included multiple planting dates between March 11 and June 18 of each year for a total of 17 individual test sets. This variation in planting dates exposed the varieties to different temperatures during the flowering period, during which maximum daily temperatures varied from 27 to 35 °C. Measurements included planting and emergence dates, days to half-inch internode elongation and 50% heading, and grain yield. Weather data, including daily maximum and minimum temperature, rainfall, solar radiation, relative humidity, and wind speed, were obtained from the Integrated Agricultural Information and Management System (iAISM) [41].

2.2. Crop Model Modifications

Three model versions were evaluated, the default version ORYZA-V1, which included a modification for the non-linear temperature dependency of plant development; ORYZA-V2, which added a modified heat stress component; and ORYZA-V3 which added a coupled leaf gas exchange methodology along with the changes in ORYZA-V2. The original ORYZA phenology model used a bilinear temperature relationship to predict development rates (Equations (A1)–(A3)). We replaced this relationship with a normalized beta function (Equation (A11), which was previously shown to more accurately simulate the response of growth duration over a broader temperature range [42,43]. This version of the model was used as the default (ORYZA-V1). Prior to model calibration, cardinal base temperatures for early leaf expansion rates were set to 10 °C for japonica rice varieties (Cocodrie and Jefferson) and 12 °C for indica rice (IR30) to account for differences in U.S. production environment [44,45].
The original ORYZA photosynthesis routine (Equations (A4)–(A6)) used hourly leaf-level predictions for sunlit/shaded leaves at three depths in the canopy using absorbed radiation and light use efficiency values [20]. This method was previously replaced by the authors with a coupled leaf-level gas exchange model within an energy balance (Equations (A12)–(A20)) at the leaf surface [33]. In this approach, leaf net photosynthesis is simulated with the Farquhar, von Caemmerer, and Berry (FvCB) biochemical model [46], which is influenced by direct inputs for solar radiation, leaf temperature, wind speed, leaf surface CO2, and relative humidity. These equations are linked with the Ball-Woodrow-Berry model for the stomatal conductance [47]. Both models are nested within an energy balance which combines the output from the stomatal conductance model to estimate leaf temperature and transpiration. These three models represent a system of interdependent equations which are solved iteratively for leaf temperature, TL (Equation (A20)). Photosynthesis and stomatal conductance are re-computed at each step until the iteration converges on a final value of TL. A detailed description can be found in [33] and supporting materials (Equations (A12)–(A20)). Photosynthetic values for the FvCB model were originally obtained using SPAR chamber data for the CLXL745 rice hybrid at different developmental stages under ambient CO2 (410 ppm) at a 28/23 °C day/night thermoperiod. The same parameter values were used for all cultivars in this study on the assumption that such responses were conserved across varietal lines.
The potential number of spikelets per rice plant is reduced by high or low thermal stress. The original high-temperature stress relationship in the ORYZA model was simulated as an exponential function of average daily maximum temperature (Equation (A10)) during the flowering period. Subsequent versions of this approach have been added as user-selected options by other researchers. For example, van Oort et al. [22] estimated both the hour during the day when flowering occurs (tpeakfl) and the air temperature at this time (Tair(tpeakfl)) as in Equations (1) and (2):
t p e a k f l = t s u n r i s e + 12.7 0.348 × T m i n 7
T a i r ( t p e a k f l ) = T m i n + ( T m a x T m i n ) × sin ( π × t p e a k f l t s u n r i s e D L + 2 × 1.5 )
where tsunrise is estimated time of sunrise calculated as the difference of 12—half of the daylength (DL) period, Tmin7 is the average daily minimum temperature, Tmin, of the preceding seven days, and Tmax is the daily maximum temperature.
This approach [18] does not account for potential genetic variation with respect to heat tolerance among varieties. These relationships were thus newly modified as part of the current study with a function obtained from a new statistical analysis of experimental data from [48] and parameterized to the following:
H E A T T T = max ( f e f ( T a i r ( t p e a k f l ) T c ) ,   0 )
S h = min ( 1.0 , exp ( 0.65 H s × H E A T T T ) 1 exp ( 0.65 H s × H E A T T T )   )
where HEATTT is the cumulative heating degree-days since anthesis, Tair is the hourly air temperature during peak flowering time tpeakfl as defined earlier, f and ef are the dates of flowering and end of flowering, Tc is a cultivar dependent critical temperature for heat stress during the flowering period,   H s   is a heat sensitive factor ranged from 0.1 to 0.25 (value of 0.167 is applied in this study) for tolerant and sensitive cultivars respectively (Figure S1), and Sh is the fraction of florets which form grain.

2.3. Model Calibration and Evaluation

Calibration values were obtained for ORYZA-V1 (Table S2) and used for all three versions. Five varieties were calibrated, three (IR30, Jefferson, and Cocodrie) using the SPAR data and two (Wells and XL753) from field data. All phenological parameters, including development rates (Table S2), were obtained using the calibration program “pheno_opt_rice2” [49]. An independent set of SPAR data from 1988 (not shown in Table S1) with day/night temperature treatments from 28/21 to 40/33 °C was used to obtain IR30 calibration parameters. An independent set of SPAR data from [6], which covered temperature treatments from 23/19 °C to 35/31 °C, was used to develop calibration values for Cocodrie and Jefferson with the assumption that tropical japonica subspecies would have similar phenological responses. Growth parameters describing biomass partitioning and specific leaf area were fixed to those contained within the ORYZA standard crop file. Spikelet growth factor (SPGF), which sets the potential number of spikelets that can flower, was estimated from observed grain yield data under a control temperature treatment of 28/25 °C at ambient CO2 from [6,34,35] for each cultivar. Critical temperatures for heat stress on spikelet fertility (Table S2) were obtained from [18,50]. Cultivars CXL753 and Wells were calibrated using 2013 field data from [40], which was also used to calibrate heat stress thresholds.
Simulations were conducted for all datasets in Table S1 to evaluate the extent to which the temperature stress and gas exchange subroutine modifications were able to improve the ability of ORYZA to estimate the response of yield to rising temperatures and CO2. Simulations were subsequently conducted for the field data in Table S1. Rice was simulated without limitation of irrigation and fertilization as per literature reports for both the controlled environment and field publications. Evaluation metrics included the ratio of simulated to observed response as a measure of relative error; root mean square error (RMSE is in Equation (A21)), and Wilmott’s index of agreement (d) [51] in Equation (A22).

3. Results

3.1. Evaluation with SPAR Chamber Data

Only ORYZA-V3 and V1 above-ground biomass predictions were compared because V2 and V1 version responses were nearly identical. Results varied according to cultivar (Table 1). The ratio of simulated to observed values, an indicator of relative error, was similar between model versions for IR30 (<5% error on average), and Wilmott’s index of agreement (d) was nearly identical (Table 1). However, RMSE was about 380 kg ha1 higher for ORYZA-V3. This error was associated with an overestimated biomass for the 1990 660 ppm 28/21 °C treatment. While V3 generally simulated a stronger biomass response with elevated CO2, there was not always a consistent pattern with temperature. In contrast, RMSE was less for ORYZA-V3 by as much as 1300 kg ha1 for Cocodrie and 190 kg ha1 for Jefferson. Both model versions over-predicted aboveground biomass at the 36 °C treatments for these two cultivars, suggesting the response of vegetative growth, such as leaf area expansion, at high temperatures may be over-emphasized.
The ORYZA-V1 version accurately simulated the IR-30 cultivar yield response to temperature and CO2 when maximum temperatures were below 32 °C. However, yields were over-estimated above 32 °C by as much as 87% (Table 2). Performance was identical with ORYZA-V2 for all data sets at the lower temperature ranges. However, the observed declines in grain yield above 32 °C were more accurately simulated with about 6% relative error. Yields for most other temperature treatments and cultivars were under-predicted by both model versions. The ORYZA-V3 model replicated grain yield more accurately than either -V2 or -V1 model versions. There was a 10% or larger increase in the agreement index compared with -V1 and a lower RMSE between 400 to about 800 kg ha1. There was little difference between ORYZA-V2 and -V3 yield predictions for Cocodrie, but modest improvements in RMSE between 140 to 270 kg ha1 were observed for both IR30 and Jefferson. As with above-ground biomass simulations, no consistent pattern was observed with CO2 and temperature in terms of model accuracy (for temperature treatments below 32 °C). However, ORYZA-V3 tended to predict higher yield values than -V2 or -V1 versions, especially for Cocodrie and Jefferson.
Yield sensitivity to the temperature at two CO2 levels is illustrated in Figure 1. All model versions tracked the differences in temperature response for cultivars below 34, 36, and 37 °C. At these higher temperatures, substantial over-predictions from ORYZA-V1 were evident. All model versions also showed a positive response to CO2 enrichment, demonstrated by the observed values. However, ORYZA-V3 exhibited more accurate changes to elevated CO2 at a given temperature treatment. For example, there were paired CO2 treatments at 28 °C and 31 °C for IR30 (Figure 1a), the results of which showed smaller relative errors than ORYZA-V2 or -V1 (Table 2). A similar response was observed among model versions for Cocodrie and Jefferson with a paired CO2 treatment at 28 °C, in which ORYZA-V3 also showed reduced relative error for the elevated CO2 response. Overall, simulations from ORYZA-V3 were closer to the observed yield than ORYZA-V2 for all varieties (Figure 1, Table 2).
Data from one of the 1987 IR30 studies [52], which included two planting dates (Table S1), were used to evaluate simulated CO2 responses from 160 to 900 ppm at 31/31 °C (Figure 2). ORYZA-V1 and ORYZA-V3 were compared for this case because V2 and V1 predictions were identical at this temperature level. ORYZA-V3 more accurately captured the response of yield to the broader range of CO2 values. Relative errors were between −7% and 9% across the 330 to 900 ppm range for ORYZA- V3 compared with under-estimates of −8 to −20% for ORYZA-V1 (Table 3). Errors were greater for the ORYZA-V3 model, however, at sub-ambient CO2 levels (<250 ppm). Across all CO2 levels, the index of agreement was higher (0.94 versus 0.91) and RMSE lower (872 versus 1065 kg ha1) for ORYZA-V3 versus ORYZA-V1, respectively. This suggested that the leaf-coupled gas exchange methodology was more accurate for projecting yields under most CO2 levels.

3.2. Evaluation with Field Data

Models were evaluated for Wells (Table 4) and CLXL753 (Table 5) cultivars using four years of field data, each with multiple planting dates. Calibration results for the four planting dates in 2013 show better metrics for ORYZA-V1 and -V2 than V3. This was due to the calibration values for -V1 being used for all model versions. However, evaluation data sets showed the opposite trend, where simulated yields were more accurate with increasing ORYZA versions. For example, RMSE decreased from a high of 2089 kg ha1 for ORYZA-V1 to a low of 1530 kg ha1 for -V3 for the Wells cultivar and an even larger reduction from 2429 to 955 kg ha1 for CLXL753. The improvements in simulation for the Wells cultivar were primarily associated with dates in 2012 and 2014, where maximum temperature values during the anthesis period were 31 °C or lower (Table 4). Given that these values were below the critical temperature threshold for heat sterility stress of 33.7 °C (Table S2), results for versions -V1 and -V2 were similar. However, the largest simulation improvements were associated with V3. For four of the treatments (5 June 2014, 3 April 2015, 21 April 2015, and 3 June 2015), ORYZA-V3 over-predicted yields compared with other versions for reasons which did not appear temperature related. Similar results were observed for CLXL753, except that ORYZA-V3 generally showed less error for 10 of the 13 sets (Table 5).

4. Discussion

Metrics for above-ground biomass and yield generally improved for ORYZA-V2 and ORYZA-V3 versus ORYZA-V1 across most temperatures and CO2 concentrations. The largest improvements, particularly for RMSE, occurred between V1 and V2, with incrementally fewer improvements between V2 and V3. This indicates the importance of using an improved heat stress algorithm for both current and future climate assessments on rice productivity. For example, ORYZA-V2, with the heat stress modification, resulted in an averaged RMSE reduction of 520 kg ha1 versus the V1 model for grain yield averaged across all SPAR data (Table 2). An average reduction of 499 kg ha1 was also observed across all field data (Table 4 and Table 5). Improvements in RMSE for ORYZA-V3, which included the heat stress modification plus the leaf-level coupled gas exchange method, were an additional 127 kg ha1 and 240 kg ha1 for SPAR and field data, respectively. The simulation of grain yield in ORYZA is primarily driven by potential spikelet number and less by the availability of assimilate, at least over the range of conditions typically used for evaluating the rice model [20]. The improvement for the ORYZA-V2 version versus V1 observed for the field data also indicates that the occurrence of heat stress during anthesis is a characteristic of the current climate in the U.S. Mississippi Delta region. Thus, improving the simulation of heat stress on spikelet fertility and potential grain number may be more important than the methodology used for carbon assimilation alone, especially if episodic high heat events during anthesis and/or grain-fill periods continue to be problematic in rice growing regions [53].
Experimental data was insufficient to thoroughly evaluate interactions with CO2 and temperature. However, the modifications implemented in ORYZA-V3 were effective in improving performance metrics for grain yield in about 80% of all treatments studied. These ranged from daytime temperatures between 24 and 36 °C and CO2 levels from 330 through 770 ppm. This range of improvement also included field data in which CO2 levels were at ambient levels, indicating that the gas exchange methodology is effective for today’s climate as well as the future. Few other studies have directly tested the effectiveness of different methods for simulating gas exchange processes for rice in response to climate factors. Differences among yield predictions among multiple rice models to changes in temperature and CO2 were observed by Li et al. [15] but were not attributed to specific carbon assimilation methods. In contrast, Hasegawa et al. [54] reported greater responsiveness to CO2 for models that used a biochemical approach for modeling photosynthesis, such as that used in the ORYZA-V3. Previous work with this approach showed how the methodology could simulate reduced stomatal conductance in response to rising CO2 concentration. This, in turn, decreases transpiration, increases leaf and canopy temperature (as much as 0.5 °C, not shown), and exerts an additional influence on photosynthesis [33]. However, despite these improvements, the ORYZA-V3 model over-estimated vegetative biomass for some SPAR datasets (Table 1), which may reflect the need to also account for photosynthetic acclimation [26], substrate-induced feedback inhibition [55,56], and/or the influence of nitrogen and light attenuation in the canopy on photosynthetic properties [31].
All model versions were able to capture differences in cultivar sensitivity to temperature. This includes the relative broad temperature response of IR30 in which grain yield formed as high as 37 °C (Table 2, Figure 1). Conversely, no yields were observed at 36 °C above for Cocodrie and Jefferson. ORYZA-V2 and V3 were consistently more accurate than the default version in terms of replicating these different temperature responses. Given that these varieties were from different rice sub-populations (e.g., IR30 was from indica and Jefferson and Cocodrie from tropical japonica), this result highlighted the need to consider heat sensitivity differences among cultivars (Table S2). The broad range of planting dates in field data resulted in the exposure of Wells and CLXL753 cultivars to maximum average daily temperatures during the flowering period ranging from 27.6 to 34.7 °C (Table 4 and Table 5). The ORYZA-V3 and -V2 models showed significant improvement compared with ORYZA-V1 in mimicking these yields. The ORYZA-V1 used a daily maximum temperature [20] as the threshold for spikelet fertility which likely resulted in an overestimation of the impact of heat stress for several field treatments for both varieties. The rice flowers often open in the morning hours [57,58] when the air temperature is cooler than the daily maximum. Thus, the improved heat sterility model used in this study, which estimated the temperature at flowering time, was shown to predict heat-stress yield reduction more accurately under both controlled environments, where square wave temperature profiles were used (Table 2), as well as more natural conditions in which temperatures varied diurnally (Table 4 and Table 5).
A strong linear relationship is known to exist between the quantity of dry matter produced during the growth period between panicle initiation through anthesis and to spikelet and grain number [20,59]. Factors associated with this response are likely related to temperature, as a driving factor of developmental rate, and solar radiation. Comparisons between simulated yields and growth stage durations for field-grown cultivars CLXL753 and Wells show high linear correlations (Table S3). Similarly, total seasonal solar radiation accumulation was also positively correlated with yields. ORYZA-V3 and V2 exhibited stronger correlations than -V1, suggesting that both the heat sterility improvement and gas exchange methodologies are important in terms of capturing expected agronomic relationships. Correlations were not significant between simulated yields and growth duration or solar radiation for the IR30, Cocodrie, or Jefferson cultivars (Table S3). This lower response may be associated with the narrower range of diurnal temperatures and the similar quantity of solar radiation available across most of the SPAR chamber treatments.
All model versions over-estimated aboveground biomass at air temperatures above 36 °C (Table 1). This was likely due to over-predictions of leaf area index as reported by van Oort et al. [22], who suggested modelers consider reduced assimilate partitioning to the leaves at lower or higher temperatures. Nagai and Makino [60] also observed differences in the ratio of whole plant leaf area to dry weight when grown under different temperatures. This suggests that specific leaf area values may need to be modified based on growth temperature to properly account for leaf area expansion and associated carbohydrate requirements. Differences in growth stage thermal sensitivities will be particularly important to account for in terms of evaluating the effectiveness of changing planting dates as a heat stress avoidance measure [61]. Further investigation into the possibility of different temperature stress thresholds associated with different rice developmental stages, as suggested by Samejima et al. [62], may also be warranted.

5. Conclusions

Improvements to heat sterility modules in rice models are crucial, and differences in varietal temperature threshold responses need to be accounted for. Incorporation of the simulation of gas exchange rates using a coupled leaf-level approach within a leaf energy balance substantially improved end-of-season biomass and improved grain yield simulations in both growth chamber and field studies across five different cultivars for two different rice subpopulations. These methods more accurately replicated responses compared with the original model approaches over a broad temperature range at two CO2 levels and thus can increase confidence when applying this modified ORYZA model for climate assessment work. The improved ORYZA model was also able to capture the average yield response of multiple cultivars to shifting planting dates across several years of field data in which cultivars were thus exposed to different thermal regimes during flowering. Results highlight the effect these improvements have on increasing the accuracy of rice model predictions to current and future climate impacts, which can serve as foundational pieces for improvements to other crop models. Our modifications to the ORYZA model can be easily adapted to other models without substantially affecting the model structure.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/agronomy12122927/s1, Figure S1: The measured response of spikelet fertility (%) to hourly temperature during anthesis for heat tolerant, moderate and susceptible cultivars as measured from Matsui et al. (2001), and (b) the modeled relationship between spikelet fertility (Sh, 0-1) to accumulated heat stress for heat tolerant and sensitive cultivars as simulated with main text equations (Equations (3) and (4)); Table S1: Data sources and experiments conducted in SPAR units used to evaluate different model versions. Experiments included three cultivars subjected to treatments with different day/night air temperature (T), CO2 concentration, and/or paddy water temperatures. Some experiments were repeated with separate planting dates as indicated; Table S2: Calibration values for cultivars IR30, Cocodrie, Jefferson, Wells, and CLXL753. Values for IR30, Cocodrie, and Jefferson were obtained from the 1988 experiment at University of Florida, and Cocodrie from the 28 and 24°C ambient CO2 treatments conducted at USDA-ARS (Table 1); Table S3: Correlation coefficients from linear regression between simulated yield versus cumulative growing season solar radiation and for different phenological stage durations for the five cultivars.

Author Contributions

Conceptualization, D.H.F. and S.L.; methodology, S.L.; validation, S.L., W.S. and Z.W.; formal analysis, D.H.F., S.L., D.T. and J.B.; resources, V.R.R.; writing—original draft preparation, S.L. and D.H.F.; writing—review and editing, D.T., J.B. and V.R.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability Statement

The experimental data presented in this study are available from the citations associated with each data source as indicated in the main text, supplemental information, and the Appendix A. Simulated data are all presented in this study as indicated. Novel computer code changes are provided as equation numbers through the main text and Appendix A.

Acknowledgments

This work was supported by the United States Department of Agriculture—Agricultural Research Service (USDA-ARS) CRIS-8042-11660-001-00D. Mention of a trademark or proprietary product does not constitute a guarantee or warranty of the product by the US Department of Agriculture and does not imply its approval to the exclusion of other products that also can be suitable. USDA is an equal opportunity provider and employer. All experiments complied with the current laws of the United States, the country in which they were performed.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

CO2atmospheric carbon dioxide concentration
dWilmott’s index of agreement;
LAIleaf area index
PARphotosynthetic active radiation
RMSEroot mean square error
RUEradiation use efficiency
SPARsoil-plant-atmosphere-research growth chambers
SPGFspikelet growth factor

Appendix A

Appendix A.1. Original ORYZA Model

Appendix A.1.1. Development

ORYZA [20] integrates growth and developmental rates at a daily time step, but hourly climate data are estimated for certain functions. Development rate has a bilinear response to temperature over hourly time periods. Hourly temperatures (Th, °C) are estimated by a sinusoidal function of daily minimum (Tmin, °C) and maximum (Tmax, °C) air temperatures:
T h   = ( T m a x + T m i n ) 2 + ( ( T m a x T m i n ) × cos ( 0.2618 ( h 14 ) ) 2
where h is the time of day. Hourly increments in heat units (HUH, °Cd h−1) are given by Equation (A2):
H U H = { 0                                                                                   T h T b a s e   o r   T h T h i g h ( T h T b a s e   )   24                                                                             T b a s e T h T o p t ( T o p t ( T h T o p t ) × ( T o p t T b a s e ) ( T h i g h T o p t ) ) / 24                                     T o p t T h T h i g h          
where Tbase, Topt, and Thigh are the base, optimal, and maximum temperature for development. The daily increment in heat units (HU, °Cd d−1) is then calculated as
HU = h = 1 24 ( H U H )
Four developmental stages are simulated (emergence, panicle initiation, flowering, and physiological maturity), the progression of which primarily depends on cumulative heat units which vary among cultivars.

Appendix A.1.2. Photosynthesis

Hourly leaf photosynthesis is based on an estimate of absorbed solar radiation and light use efficiency. This rate is calculated separately for a single shaded and sunlit leaf at each of three depths in the canopy using Equations (A4)–(A6):
A n = A m ( 1 exp ( ε I a A m ) ) R d
A m = ( 49.57 34.26 ) × ( 1 exp ( 0.208 ( C O 2 60 ) 49.57 ) )
ε = ε 340 ppm ( 1 exp ( 0.00305 C O 2 0.222 ) ) ( 1 exp ( 0.00305 × 340 0.222 ) )
where An is the net CO2 assimilation rate (kg CO2 ha−1 leaf h−1), Am is the CO2 assimilation rate at light saturation, ε is the initial light-use efficiency (kg CO2 ha−1 h−1 (J m2 s)1), and other variable and units were defined in Table A1. Note that ε340 ppm is ε at a CO2 concentration of 340 ppm and linearly declines from a maximum of 0.54 when average temperatures rise above 10 °C as per [63]. In these equations, constant parameter values are used and assumed to be conserved across rice cultivars.
Leaf net photosynthesis is scaled up to the canopy level using a sunlit/shaded leaf approach. At each depth in the canopy, absorbed flux for diffuse, direct, and total PAR is computed for each leaf type using an exponential decay function for direct beam solar radar radiation as in Equation (A7) based on [64]:
I a = ( 1 ρ ) I 0 exp ( k × L )
where Ia (J m2 ground s1) is the net photosynthetically active radiation (PAR) at depth L in the canopy, I0 (J m2 ground s1) is the photosynthetically active radiation at the top of the canopy, L is cumulative LAI (m2 leaf m2 ground), ρ is the light reflection coefficient of the canopy, and k is the canopy light extinction coefficient for PAR. Assimilation rate and absorbed radiation are integrated using a three-point Gaussian method at each canopy layer which is then scaled up to estimate the daily whole canopy net assimilation rate. Photosynthesis is allocated to the leaf, stem, and panicle with carbon partitioning coefficients that vary with plant phenology and water or nitrogen stress.

Appendix A.1.3. Temperature Stress

ORYZA includes the option for users to enter an observed spikelet number as an input value. If this option is used, heat sterility functions are not utilized, as this number is assumed to reflect the potential grain number. However, in the more typical case where spikelet number must be simulated in the model, the following methods are used to account for cold and hot temperature stresses.
Spikelet sterility is affected by cumulative daily average temperatures below 22 °C prior to anthesis as in Equations (8) and (9):
C O L D T T = p f ( 22 T d )
S c = 1 ( 4.6 + 0.054 × C O L D T T 1.56 ) / 100
where COLDTT is cumulative cold temperature degree-days (°Cd), Td is the daily average air temperature, p and f are the dates of panicle initiation and flowering, and Sc is the percentage sterility caused by cold.
Spikelet fertility is also reduced when averaged Tmax at the flowering period is above a critical value as in Equation (A10), where the fraction of fertile rice spikelets reduced by heat (Sh) is estimated as:
S h = 1 / ( 1 + exp ( 0.853 ( T m a x 36.6 ) )
The actual stress factor on spikelet sterility at anthesis is calculated as the minimum of these two stresses and reduces the number of spikelets which are then associated with grain number and yield.

Appendix A.2. Modified ORYZA Model

Appendix A.2.1. Development

The original bilinear model was replaced by a beta function (Equation (A11)) to more accurately estimate hourly air temperatures. The normalized beta model ([65]) is given by the following equation:
H U H = { ( T h T b a s e T o p t T b a s e ) ( T h i g h T h T h i g h T o p t ) ( ( T h i g h h ) / ( T o p t T b a s e ) ) } T S E N
where HUH, Th, Tbase, Topt (°C), and Thigh were previously defined and TSEN determines the curvature of the response.

Appendix A.2.2. Coupled Gas Exchange Model

The three sub-models (FvCB, BWB versions, and energy balance) were interdependent and solved numerically using the Newton–Raphson method [66] through a nested iterative procedure developed by [67]. This modification simulated the direct effects of CO2 on stomatal closure and subsequent increases in leaf temperature which in turn influenced photosynthesis and transpiration.

Leaf Photosynthesis

The Farquhar, von Caemmerer, and Berry (FvCB) [46] biochemical model of leaf photosynthesis as modified by [68] was used. The three rate-limiting steps in the FvCB model for net leaf photosynthesis (An) were the ribulose 1·5-bisphosphate carboxylase/oxygenase (Rubisco) rate (Ac), the ribulose-1,5-bisphosphate (RuBP) regeneration/electron transport-limited rate (Aj), and the triose phosphate utilization (Tp)-limited rate ( A P ) (Equation (A12) through Equation (A15)). Other inputs included the intercellular CO2 partial pressure (Ci) estimated as a fraction of ambient CO2 partial pressure.
A n = min ( A c , A j , A P )
A c = ( C i Γ * ) V c m a x C i + K c ( 1 + O K o ) R d
A j = ( C i Γ * ) J 4 ( C i + 2 Γ * ) R d
A P = 3 T p
C i = C a × 0.7
J = σ I a + J m a x ( σ I a + J m a x ) 2 4 θ I a J m a x 2 θ
Parameter values used in this study were obtained from [33]. Temperature response for Vcmax, Jmax and T p were expressed in Equation (A18), where c is a scaling constant, Δ H a is the enthalpy of activation, TL is leaf temperature in °C and R is the universal gas constant (8.314 J mol1 K1), and p a r a m e t e r 25 is the value of a parameter at 25 °C, Δ H d is the enthalpy of deactivation and Δ S is entropy. An estimate of each parameter at 25 °C was obtained using the method of [69].
T p = p a r a m e t e r 25 × e ( c Δ H a R × ( 273.15 + T L   ) )   1 + e ( Δ S × ( 273.15 + T L ) Δ H d R × ( 273.15 + T L   ) )  

Stomatal Conductance

The Ball–Woodrow–Berry model (BWB) was used to simulate leaf stomatal conductance to water vapor [47]:
g s = g 0 + g 1 A n h s C s
where g s   is stomatal conductance (µmol m2 s1), hs is humidity at the leaf surface, Cs is leaf surface CO2 concentration, g 0 is the minimum conductance (mol m2 s1), and g 1 is a slope parameter. Parameter values used in this study were time invariant with g 0 at 0.10 and g 1 at 8.20.

Leaf Energy Balance Model

Leaf temperature (TL) was determined by a linear solution of the leaf energy budget [70]:
T L = T h + R a b s ε σ T h 4 λ g v V P D / P a c p ( g h + g r ) + λ ( ( d e s ( T h ) d T ) / P a ) g v
where T h (°C) is hourly air temperature, R a b s (W m−2) is absorbed long wave and short-wave radiation per surface leaf area, ε   (0.97) is leaf thermal emissivity, σ   (5.67 × 10−8, W m−2 K−4) is Stefan–Boltzman constant per surface area, λ   (44, kJ mol−1) is the latent heat of vaporization at 25 °C, g v (mol m−2 s−1) is total water vapor conductance per surface leaf area, VPD (kPa) is vapor pressure deficit of ambient air, P a (kPa) is atmospheric pressure, c p (J mol−1 C−1) is specific heat of air, g h (mol m−2 s−1) is heat conductance for boundary layer per surface leaf area, g r (mol m−2 s−1) is radiative conductance per surface leaf area, and e s (kPa) is vapor pressure at leaf surface, and g b (mol m−2 s−1) is boundary layer conductance to water vapor.
Root mean square error and Wilmott’s index of agreement [51] were given in Equations (A21) and (A22).
R M S E = N 1 i = 1 N ( O i S i ) 2
d = 1 i = 1 N ( S i O i ) 2 i = 1 N ( | S i O ¯ | + | O i O ¯ | ) 2 ,     0 d 1
where, N is the number of observations, and Oi and Si are observed and simulated values for observation I, and d is the agreement index value.
Table A1. Equation variables and units.
Table A1. Equation variables and units.
VariableDescriptionUnitEquation Number
Ancanopy or leaf net photosynthetic rateµmol m−2 s−1(A4), (A12) and (A19)
Iaphotosynthetically active radiation incident to leaf surfaceµmol m−2 s−1(A4), (A7)
εlight use efficiencykg CO2 ha−1 h−1 (J m2 s−1)(A4), (A6)
AmCO2 assimilation rate at light saturationkg CO2 ha−1 leaf h−1(A4), (A5)
R d day respirationµmol m−2 s−1(A4), (A13) and (A14)
I0photosynthetically active radiation at the top of the canopyJ m−2 ground s−1(A7)
Lcumulative LAIm2 leaf m−2 ground(A7)
ρ light reflection coefficient of the canopy (A7)
kcanopy light extinction coefficient for PAR (A7)
AcRubisco carboxylation-limited rateµmol m−2 s−1(A12), (A13)
A P triose phosphate utilization (Tp)-limited photosynthetic rateµmol m−2 s−1(A12), (A15)
AjRuBP regeneration-or electron transport-limited rateµmol m−2 s−1(A12), (A14)
CiIntercellular CO2 concentrationµbar(A13), (A16)
Γ*the CO2 compensation point in the absence of Rdµmol m−2 s−1(A13)
Vcmaxmaximum carboxylation rateµmol m−2 s−1(A13)
KcMichaelis constant of Rubisco affinity for carbon dioxidekPa(A13)
Opartial pressure of oxygen at RubiscokPa(A13)
KoMichaelis constant of Rubisco affinity for carbon dioxidekPa(A13)
Jphotosystem (PS) II electron transport rateµmol m−2 s−1(A17)
TP: triose phosphate utilizationµmol m−2 s−1(A15), (A18)
Caambient CO2 concentrationµbar(A16)
Jmaxpotential maximum electron transport rateµmol m−2 s−1(A17)
σelectron transport efficiency of PS ΙΙ(A17)
θ a curvature parameter(A17)

References

  1. Sandhu, N.; Kumar, A. Bridging the rice yield gaps under drought: QTLs, genes, and their use in breeding programs. Agronomy 2017, 7, 27. [Google Scholar] [CrossRef] [Green Version]
  2. Tripathi, A.; Tripathi, D.K.; Chauhan, D.; Kumar, N.; Singh, G. Paradigms of climate change impacts on some major food sources of the world: A review on current knowledge and future prospects. Agric. Ecosyst. Environ. 2016, 216, 356–373. [Google Scholar] [CrossRef]
  3. Wang, D.R.; Bunce, J.A.; Tomecek, M.B.; Gealy, D.; McClung, A.; McCouch, S.R.; Ziska, L.H. Evidence for divergence of response in Indica, Japonica, and wild rice to high CO2 × temperature interaction. Glob. Change Biol. 2016, 22, 2620–2632. [Google Scholar] [CrossRef] [PubMed]
  4. Xu, Y.; Chu, C.; Yao, S. The impact of high-temperature stress on rice: Challenges and solutions. Crop. J. 2021, 9, 963–976. [Google Scholar] [CrossRef]
  5. Hasegawa, T.; Sakai, H.; Tokida, T.; Usui, Y.; Yoshimoto, M.; Fukuoka, M.; Nakamura, H.; Shimono, H.; Okada, M. Rice free-air carbon dioxide enrichment studies to improve assessment of climate change effects on rice agriculture. Advancesin 2016, 7, 45–68. [Google Scholar] [CrossRef]
  6. Baker, J.T. Yield responses of southern US rice cultivars to CO2 and temperature. Agric. For. Meteorol. 2004, 122, 129–137. [Google Scholar] [CrossRef]
  7. Lv, C.; Huang, Y.; Sun, W.; Yu, L.; Zhu, J. Response of rice yield and yield components to elevated [CO2]: A synthesis of updated data from FACE experiments. Eur. J. Agron. 2019, 112, 125961. [Google Scholar] [CrossRef]
  8. Cai, C.; Yin, X.; He, S.; Jiang, W.; Si, C.; Struik, P.C.; Luo, W.; Li, G.; Xie, Y.; Xiong, Y.; et al. Responses of wheat and rice to factorial combinations of ambient and elevated CO2 and temperature in FACE experiments. Glob. Chang. Biol. 2015, 22, 856–874. [Google Scholar] [CrossRef]
  9. Ewert, F.; Rötter, R.; Bindi, M.; Webber, H.; Trnka, M.; Kersebaum, K.; Olesen, J.; van Ittersum, M.; Janssen, S.; Rivington, M.; et al. Crop modelling for integrated assessment of risk to food production from climate change. Environ. Model. Softw. 2015, 72, 287–303. [Google Scholar] [CrossRef]
  10. White, J.W.; Hoogenboom, G.; Kimball, B.A.; Wall, G.W. Methodologies for simulating impacts of climate change on crop production. Field Crop. Res. 2011, 124, 357–368. [Google Scholar] [CrossRef]
  11. Stella, T.; Webber, H.; E Olesen, J.; Ruane, A.C.; Fronzek, S.; Bregaglio, S.; Mamidanna, S.; Bindi, M.; Collins, B.; Faye, B.; et al. Methodology to assess the changing risk of yield failure due to heat and drought stress under climate change. Environ. Res. Lett. 2021, 16, 104033. [Google Scholar] [CrossRef]
  12. Jägermeyr, J.; Müller, C.; Ruane, A.C.; Elliott, J.; Balkovic, J.; Castillo, O.; Faye, B.; Foster, I.; Folberth, C.; Franke, J.A.; et al. Climate impacts on global agriculture emerge earlier in new generation of climate and crop models. Nat. Food 2021, 1–13. [Google Scholar] [CrossRef]
  13. Bezner Kerr, R.; Hasegawa, T.; Lasco, R.; Bhatt, I.; Deryng, D.; Farrell, A.; Gurney-Smith, H.; Ju, H.; Lluch-Cota, S.; Meza, F.; et al. Food, Fibre, and Other Ecosystem Products, in Climate Change 2022: Impacts, Adaptation and Vulnerability. In Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change; Pörtner, H.O., Roberts, D.C., Tignor, M., Poloczanska, E.S., Mintenbeck, K., Alegría, A., Craig, M., Langsdorf, S., Löschke, S., Möller, V., et al., Eds.; Cambridge University Press: Cambridge, UK; New York, NY, USA, 2022; pp. 713–906. [Google Scholar]
  14. Maiorano, A.; Martre, P.; Asseng, S.; Ewert, F.; Müller, C.; Rötter, R.P.; Ruane, A.C.; Semenov, M.A.; Wallach, D.; Wang, E.; et al. Crop model improvement reduces the uncertainty of the response to temperature of multi-model ensembles. Field Crop. Res. 2017, 202, 5–20. [Google Scholar] [CrossRef] [Green Version]
  15. Li, T.; Hasegawa, T.; Yin, X.; Zhu, Y.; Boote, K.; Adam, M.; Bregaglio, S.; Buis, S.; Confalonieri, R.; Fumoto, T.; et al. Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Glob. Chang. Biol. 2014, 21, 1328–1341. [Google Scholar] [CrossRef] [PubMed]
  16. Toreti, A.; Deryng, D.; Tubiello, F.N.; Müller, C.; Kimball, B.A.; Moser, G.; Boote, K.; Asseng, S.; Pugh, T.A.M.; Vanuytrecht, E.; et al. Narrowing uncertainties in the effects of elevated CO2 on crops. Nat. Food 2020, 1, 775–782. [Google Scholar] [CrossRef]
  17. Fleisher, D.H.; Condori, B.; Quiroz, R.; Alva, A.; Asseng, S.; Barreda, C.; Bindi, M.; Boote, K.J.; Ferrise, R.; Franke, A.C.; et al. A potato model intercomparison across varying climates and productivity levels. Glob. Chang. Biol. 2016, 23, 1258–1281. [Google Scholar] [CrossRef] [PubMed]
  18. Jagadish, S.; Craufurd, P.; Wheeler, T. High temperature stress and spikelet fertility in rice (Oryza sativa L.). J. Exp. Bot. 2007, 58, 1627–1635. [Google Scholar] [CrossRef] [Green Version]
  19. Hakata, M.; Wada, H.; Masumoto-Kubo, C.; Tanaka, R.; Sato, H.; Morita, S. Development of a new heat tolerance assay system for rice spikelet sterility. Plant Methods 2017, 13, 1–8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Bouman, B.A.M.; Kropff, M.J.; Wopereis, M.C.S.; ten Berge, H.F.M.; van Laar, H.H. ORZYA2000: Modeling lowland rice; International Rice Research Institute: Los Banos, Philippines; Wageningen University and Research Centre: Wageningen, The Netherlands, 2001; p. 235. [Google Scholar]
  21. Li, T.; Angeles, O.; Marcaida, M.; Manalo, E.; Manalili, M.P.; Radanielson, A.; Mohanty, S. From ORYZA2000 to ORYZA (v3): An improved simulation model for rice in drought and nitrogen-deficient environments. Agric. For. Meteorol. 2017, 237-238, 246–256. [Google Scholar] [CrossRef] [PubMed]
  22. van Oort, P.A.; de Vries, M.E.; Yoshida, H.; Saito, K. Improved climate risk simulations for rice in arid environments. PLoS ONE 2015, 10, e0118114. [Google Scholar] [CrossRef] [PubMed]
  23. Li, S.; Fleisher, D.; Timlin, D.; Reddy, V.R.; Wang, Z.; McClung, A. Evaluation of different crop models for simulating rice development and yield in the U.S. Mississippi Delta. Agronomy 2020, 10, 1905. [Google Scholar] [CrossRef]
  24. Sun, T.; Hasegawa, T.; Liu, B.; Tang, L.; Liu, L.; Cao, W.; Zhu, Y. Current rice models underestimate yield losses from short-term heat stresses. Glob. Chang. Biol. 2020, 27, 402–416. [Google Scholar] [CrossRef] [PubMed]
  25. Asseng, S.; Ewert, F.; Rosenzweig, C.; Jones, J.W.; Hatfield, J.L.; Ruane, A.C.; Boote, K.; Thorburn, P.J.; Rötter, R.; Cammarano, D.; et al. Uncertainty in simulating wheat yields under climate change. Nat. Clim. Chang. 2013, 3, 827–832. [Google Scholar] [CrossRef] [Green Version]
  26. Yin, X. Improving ecophysiological simulation models to predict the impact of elevated atmospheric CO2 concentration on crop productivity. Ann. Bot. 2013, 112, 465–475. [Google Scholar] [CrossRef] [Green Version]
  27. Negi, J.; Hashimoto-Sugimoto, M.; Kusumi, K.; Iba, K. New approaches to the biology of stomatal guard cells. Plant Cell Physiol. 2013, 55, 241–250. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Konrad, W.; Katul, G.; Roth-Nebelsick, A. Leaf temperature and its dependence on atmospheric CO2 and leaf size. Geol. J. 2020, 56, 866–885. [Google Scholar] [CrossRef] [Green Version]
  29. Boote, K.J.; Jones, J.W.; White, J.W.; Asseng, S.; Lizaso, J.I. Putting mechanisms into crop production models. Plant, Cell Environ. 2013, 36, 1658–1672. [Google Scholar] [CrossRef]
  30. Yang, Y.; Kim, S.-H.; Timlin, D.J.; Fleisher, D.H.; Quebedeaux, B.; Reddy, V.R. Simulating canopy transpiration and photosynthesis of corn plants under contrasting water regimes using a coupled model. Trans. ASABE 2009, 52, 1011–1024. [Google Scholar] [CrossRef]
  31. Fleisher, D.H.; Timlin, D.J.; Yang, Y.; Reddy, V. Simulation of potato gas exchange rates using SPUDSIM. Agric. For. Meteorol. 2010, 150, 432–442. [Google Scholar] [CrossRef]
  32. Sun, W.; Fleisher, D.; Timlin, D.; Li, S.; Wang, Z.; Reddy, V. Effects of elevated CO2 and temperature on soybean growth and gas exchange rates: A modified GLYCIM model. Agric. For. Meteorol. 2021, 312, 108700. [Google Scholar] [CrossRef]
  33. Li, S.; Fleisher, D.; Wang, Z.; Barnaby, J.; Timlin, D.; Reddy, V. Application of a coupled model of photosynthesis, stomatal conductance and transpiration for rice leaves and canopy. Comput. Electron. Agric. 2021, 182, 106047. [Google Scholar] [CrossRef]
  34. Baker, J.T.; Allen, L.H. Contrasting crop species responses to CO2 and temperature: Rice, soybean and citrus. Vegetatio 1993, 104, 239–260. [Google Scholar] [CrossRef]
  35. Baker, J.T.; Allen, L.H., Jr. Effects of CO2 and temperature on rice: A summary of five growing seasons. J. Agric. Meteorol. 1993, 48, 575–582. [Google Scholar] [CrossRef]
  36. Baker, J.; Allen, L.; Boote, K.; Jones, P.; Jones, J. Developmental responses of rice to photoperiod and carbon dioxide concentration. Agric. For. Meteorol. 1990, 50, 201–210. [Google Scholar] [CrossRef]
  37. Baker, J.T.; Allen, L.H. Rice Growth, Yield and Photosynthetic Responses to Elevated Atmospheric Carbon Dioxide Concentration and Drought. J. Crop Improv. 2005, 13, 7–30. [Google Scholar] [CrossRef]
  38. Baker, J.; Allen, L.; Boote, K. Response of rice to carbon dioxide and temperature. Agric. For. Meteorol. 1992, 60, 153–166. [Google Scholar] [CrossRef]
  39. Ruane, A.C.; Goldberg, R.; Chryssanthacopoulos, J. Climate forcing datasets for agricultural modeling: Merged products for gap-filling and historical climate series estimation. Agric. For. Meteorol. 2015, 200, 233–248. [Google Scholar] [CrossRef] [Green Version]
  40. UA-CES (University of Arkansas Cooperative Extension Service). Arkansas Rice Performance Trials; University of Arkansas Division of Agriculture, Cooperative Extension Service: Little Rock, AR, USA, 2015.
  41. Wilson, l.T.; Yang, Y.; Wang, J. Integrated Agricultural Information and Management System (iAIMS): World Climatic Data. 2015. [cited 2015 April]. Available online: http://beaumont.tamu.edu/ClimaticData/ (accessed on 20 November 2022).
  42. Kim, S.; Yang, Y.; Timlin, D.J.; Fleisher, D.H.; Dathe, A.; Reddy, V.R.; Staver, K. Modeling temperature responses of leaf growth, development, and biomass in maize with MAIZSIM. Agron. J. 2012, 104, 1523–1537. [Google Scholar] [CrossRef] [Green Version]
  43. Fleisher, D.H.; Shillito, R.M.; Timlin, D.J.; Kim, S.; Reddy, V.R. Approaches to modeling potato leaf appearance rate. Agron. J. 2006, 98, 522–528. [Google Scholar] [CrossRef] [Green Version]
  44. Rebolledo, M.C.; Dingkuhn, M.; Péré, P.; McNally, K.L.; Luquet, D. Developmental dynamics and early growth vigour in rice. I. relationship between development rate (1/Phyllochron) and growth. J. Agron. Crop Sci. 2012, 198, 374–384. [Google Scholar] [CrossRef]
  45. Sánchez, B.; Rasmussen, A.; Porter, J.R. Temperatures and the growth and development of maize and rice: A review. Glob. Chang. Biol. 2013, 20, 408–417. [Google Scholar] [CrossRef] [PubMed]
  46. Farquhar, G.D.; Von Caemmerer, S.; Berry, J.A. A biochemical model of photosynthetic CO2 assimilation in leaves of C3 species. Planta 1980, 149, 78–90. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Ball, J.T.; Woodrow, I.E.; Berry, J.A. A model predicting stomatal conductance and its contribution to the control of photosynthesis under different environmental conditions. In Progress in Photosynthesis Research; Springer: Dordrecht, The Netherlands, 1987; pp. 221–224. ISBN 978-94-017-0519-6. [Google Scholar]
  48. Matsui, T.; Omasa, K.; Horie, T. The difference in sterility due to high temperatures during the flowering period among japonica-rice varieties. Plant Prod. Sci. 2001, 4, 90–93. [Google Scholar] [CrossRef]
  49. van Oort, P.; Zhang, T.; de Vries, M.; Heinemann, A.; Meinke, H. Correlation between temperature and phenology prediction error in rice (Oryza sativa L.). Agric. For. Meteorol. 2011, 151, 1545–1555. [Google Scholar] [CrossRef]
  50. Wu, C.; Cui, K.; Li, Q.; Li, L.; Wang, W.; Hu, Q.; Ding, Y.; Li, G.; Fahad, S.; Huang, J.; et al. Estimating the yield stability of heat-tolerant rice genotypes under various heat conditions across reproductive stages: A 5-year case study. Sci. Rep. 2021, 11, 13604. [Google Scholar] [CrossRef] [PubMed]
  51. Willmott, C.J. Some comments on the evaluation of model performance. Bull. Am. Meteorol. Soc. 1982, 63, 1309–1313. [Google Scholar] [CrossRef]
  52. Baker, J.T.; Allen, L.H.J.; Boote, K.J.; Rowland-Bamford, A.J.; Jones, J.W.; Jones, P.H.; Bowes, G.; Albrecht, S.L. Response of vegetation to carbon dioxide, Ser. No. 043. In Response of Rice to Subambient and Superambient Carbon Dioxide Concentrations; U.S. Dept of Agriculture, Agricultural Research Service (in cooperation with University of Florida) for the US Dept. of Energy, Carbon Dioxide Research Division, Office of Energy Research: Washington, DC, USA, 1988. [Google Scholar]
  53. Ziska, L.H.; Fleisher, D.H.; Linscombe, S. Ratooning as an adaptive management tool for climatic change in rice systems along a north-south transect in the southern Mississippi valley. Agric. For. Meteorol. 2018, 263, 409–416. [Google Scholar] [CrossRef]
  54. Hasegawa, T.; Li, T.; Yin, X.; Zhu, Y.; Boote, K.; Baker, J.; Bregaglio, S.; Buis, S.; Confalonieri, R.; Fugice, J.; et al. Causes of variation among rice models in yield response to CO2 examined with Free-Air CO2 Enrichment and growth chamber experiments. Sci. Rep. 2017, 7, 14858. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Fleisher, D.H.; Dathe, A.; Timlin, D.; Reddy, V. Improving potato drought simulations: Assessing water stress factors using a coupled model. Agric. For. Meteorol. 2015, 200, 144–155. [Google Scholar] [CrossRef]
  56. Basu, P.; Sharma, A.; Garg, I.; Sukumaran, N. Tuber sink modifies photosynthetic response in potato under water stress. Environ. Exp. Bot. 1999, 42, 25–39. [Google Scholar] [CrossRef]
  57. Julia, C.; Dingkuhn, M. Variation in time of day of anthesis in rice in different climatic environments. Eur. J. Agron. 2012, 43, 166–174. [Google Scholar] [CrossRef]
  58. Ishimaru, T.; Hirabayashi, H.; Ida, M.; Takai, T.; San-Oh, Y.A.; Yoshinaga, S.; Ando, I.; Ogawa, T.; Kondo, M. A genetic resource for early-morning flowering trait of wild rice Oryza officinalis to mitigate high tempera-ture-induced spikelet sterility at anthesis. Ann. Bot. 2010, 106, 515–520. [Google Scholar] [CrossRef] [Green Version]
  59. Islam, M.; Morison, J. Influence of solar radiation and temperature on irrigated rice grain yield in Bangladesh. Field Crop. Res. 1992, 30, 13–28. [Google Scholar] [CrossRef]
  60. Nagai, T.; Makino, A. Differences Between Rice and Wheat in Temperature Responses of Photosynthesis and Plant Growth. Plant Cell Physiol. 2009, 50, 744–755. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  61. Espe, M.B.; Yang, H.; Cassman, K.G.; Guilpart, N.; Sharifi, H.; Linquist, B.A. Estimating yield potential in temperate high-yielding, direct-seeded US rice production systems. Field Crop. Res. 2016, 193, 123–132. [Google Scholar] [CrossRef] [Green Version]
  62. Samejima, H.; Kikuta, M.; Katura, K.; Menge, D.; Gichuhi, E.; Wainaina, C.; Kimani, J.; Inukai, Y.; Yamauchi, A.; Makihara, D. A method for evaluating cold tolerance in rice during reproductive growth stages under natural low-temperature conditions in tropical highlands in Kenya. Plant Prod. Sci. 2020, 23, 466–476. [Google Scholar] [CrossRef]
  63. Goudriaan, J.; van Laar, H.H. Simulation of Crop Growth Processes; Kluwer Academic Publishers: Dordrecht, the Netherlands, 1994. [Google Scholar]
  64. Spitters, C.J.T.; Toussaint, H.A.J.M.; Goudriaan, J. Separating the diffuse and direct component of global radiation and its im-plications for modeling canopy photosynthesis. I. components of incoming radiation. Agric. For. Meteorol. 1986, 38, 217–219. [Google Scholar] [CrossRef]
  65. Yin, X.; Goudriaan, J.; Lantinga, E.A.; Vos, J.; Spiertz, H.J. A Flexible Sigmoid Function of Determinate Growth. Ann. Bot. 2002, 91, 361–371. [Google Scholar] [CrossRef]
  66. Galantai, A. The theory of Newton’s method. J. Comput. Appl. Math. 2000, 124, 25–44. [Google Scholar] [CrossRef] [Green Version]
  67. Kim, S.H.; Lieth, J.H. A coupled model of photosynthesis, stomatal conductance and transpiration for a rose leaf (Rosa hybrida L.). Ann. Bot. 2003, 91, 771–781. [Google Scholar] [CrossRef]
  68. Harley, P.C.; Loreto, F.; Di Marco, G.; Sharkey, T.D. Theoretical considerations when estimating the mesophyll conductance to CO2 flux by analysis of the response of photosynthesis to CO2. Plant Physiol. 1992, 98, 1429–1436. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  69. Sharkey, T.D.; Bernacchi, C.J.; Farquhar, G.D.; Singsaas, E.L. Fitting photosynthetic carbon dioxide response curves for C3 leaves. Plant Cell Environ. 2007, 30, 1035–1040. [Google Scholar] [CrossRef] [PubMed]
  70. Campbell, G.S.; Norman, J.M. An Introduction to Environmental Biophysics, 2nd ed.; Springer: New York, NY, USA, 1998. [Google Scholar] [CrossRef]
Figure 1. Observed and simulated response of grain yield from three model versions versus daily temperature value under ambient (aCO2) or elevated (eCO2) CO2 for cultivars (a) IR30 and (b) Cocodrie and Jefferson using data from Table 2. Observed and simulated means for Cocodrie and Jefferson were shown as averages across varieties due to very small yield differences.
Figure 1. Observed and simulated response of grain yield from three model versions versus daily temperature value under ambient (aCO2) or elevated (eCO2) CO2 for cultivars (a) IR30 and (b) Cocodrie and Jefferson using data from Table 2. Observed and simulated means for Cocodrie and Jefferson were shown as averages across varieties due to very small yield differences.
Agronomy 12 02927 g001
Figure 2. Comparison of observed (from [52]) and simulated rice yield for ORYZA-V1 and ORYZA-V3 under varying CO2 concentration at constant day/night temperature of 31 °C averaged over two planting dates for the IR30 cultivar.
Figure 2. Comparison of observed (from [52]) and simulated rice yield for ORYZA-V1 and ORYZA-V3 under varying CO2 concentration at constant day/night temperature of 31 °C averaged over two planting dates for the IR30 cultivar.
Agronomy 12 02927 g002
Table 1. Comparison of observed (Obs) and simulated (Sim) above-ground rice biomass for ORYZA-V1 and ORYZA-V3 model versions in response to CO2 and day/night temperature (T) treatments for three rice varieties. The ratio of simulated to observed final biomass (Sim/Obs) was indicated for each individual treatment, and Wilmott’s index of agreement (d) and RMSE computed across all treatments for each variety.
Table 1. Comparison of observed (Obs) and simulated (Sim) above-ground rice biomass for ORYZA-V1 and ORYZA-V3 model versions in response to CO2 and day/night temperature (T) treatments for three rice varieties. The ratio of simulated to observed final biomass (Sim/Obs) was indicated for each individual treatment, and Wilmott’s index of agreement (d) and RMSE computed across all treatments for each variety.
VarietyYearCO2 (ppm)T (°C)Obs (kg ha−1)ORYZA-V1ORYZA-V3
Sim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/Obs
IR30198733031/3112,92515,0781.1714,5401.12
198733031/3116,92014,6780.8714,7900.87
198766031/3117,62520,8561.1822,5461.28
198766031/3121,85519,8280.9122,1401.01
198933028/2115,27516,0381.0515,9991.05
198966025/1818,33022,0051.2021,3691.17
198966028/2120,91520,8111.0022,2151.06
198966034/2719,03516,7940.8818,0740.95
198966037/3016,68513,6070.8213,2650.80
199033028/2117,83619,0321.0719,4721.09
199066028/2122,86923,6141.0327,0161.18
Average18,20618,3951.0219,2211.05
d-0.84-0.85-
RMSE (kg ha−1)-2232-2618-
Cocodrie200035028/2818,03615,1740.8415,9790.89
200070024/2424,33620,5580.8420,6030.85
200070028/2825,23619,2000.7621,6640.86
200070032/3224,04817,9180.7520,6430.86
200070036/3612,31214,2671.1614,9451.21
Average20,79417,4230.8718,7670.93
d-0.74-0.84-
RMSE (kg ha−1)-4479-3145-
Jefferson200035028/2815,87615,2240.9616,0331.01
200070024/2425,84820,6950.8020,6700.80
200070028/2821,38419,3290.9021,7381.02
200070032/3220,77218,0050.8720,7181.00
200070036/3610,15214,2711.4114,9411.47
Average18,80617,5050.9918,8201.06
d-0.82-0.84-
RMSE (kg ha−1)-3341-3159-
Table 2. Comparison of observed (Obs) and simulated (Sim) rice grain yield for ORYZA-V1, -V2, and -V3 model versions in response to CO2 and day/night temperature (T) treatments for three rice varieties. The ratio of simulated to observed final biomass (Sim/Obs) was indicated for each individual treatment, and Wilmott’s index of agreement (d) and RMSE computed across all treatments for each variety.
Table 2. Comparison of observed (Obs) and simulated (Sim) rice grain yield for ORYZA-V1, -V2, and -V3 model versions in response to CO2 and day/night temperature (T) treatments for three rice varieties. The ratio of simulated to observed final biomass (Sim/Obs) was indicated for each individual treatment, and Wilmott’s index of agreement (d) and RMSE computed across all treatments for each variety.
VarietyYearCO2 (ppm)T (°C)Obs Yield (kg ha−1)ORYZA-V1ORYZA-V2ORYZA-V3
Sim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/Obs
IR30198733031/31520039960.7739960.7741690.80
198733031/31430047441.1047441.1051121.19
198766031/31680055870.8255870.8264280.95
198766031/31640065321.0265321.0276451.19
198933028/21660053320.8153320.8156870.86
198966025/18840071700.8571700.8570720.84
198966028/2110,40072940.7072940.7080630.78
198966034/27340057261.6832230.9536271.07
198966037/30100018671.879380.949950.99
199033028/21800066170.8366170.8370770.88
199066028/2110,10087880.8787880.8799220.98
Average641857871.0354750.8859820.96
d-0.88-0.92-0.96-
RMSE (kg ha−1)-1529-1334-1062-
Cocodrie200035028/28523049580.9549580.9557401.10
200070024/24681474571.0374571.0380131.11
200070028/28782363310.8363310.8376491.00
200070032/32673358730.8458730.8474331.06
200070036/3603344 984-1104-
Average532055930.9151210.9159881.07
d-0.82-0.96-0.98-
RMSE (kg ha−1)-1446-795-829-
Jefferson200035028/28460845460.9945460.9952611.14
200070024/24723668840.9568840.9575101.04
200070028/28723658040.8058040.8070110.97
200070032/32597653910.9053910.9067711.13
200070036/3603052-934-1066-
Average501151350.9147120.9155241.07
d-0.85-0.97-0.98-
RMSE (kg ha−1)-1539-824-681-
Table 3. Comparison of observed (Obs) and simulated (Sim) rice grain yield for ORYZA-V1 and ORYZA-V3 model versions in response to varying CO2 concentrations. Data were averaged from two different experiments, Exp I and II, that varied in planting date (22 January or 23 June) from [52]. The ratio of simulated to observed yield (Sim/Obs) was indicated along with Wilmott’s index of agreement (d) and RMSE.
Table 3. Comparison of observed (Obs) and simulated (Sim) rice grain yield for ORYZA-V1 and ORYZA-V3 model versions in response to varying CO2 concentrations. Data were averaged from two different experiments, Exp I and II, that varied in planting date (22 January or 23 June) from [52]. The ratio of simulated to observed yield (Sim/Obs) was indicated along with Wilmott’s index of agreement (d) and RMSE.
ExpCO2 (ppm)Observed (kg ha−1)ORYZA-V1ORYZA-V3
Sim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/Obs
I & II16034002386 ± 4300.701547 ± 6150.45
I & II25041003629 ± 8370.893564 ± 6470.87
I & II33048004426 ± 9480.924840 ± 6081.01
II only50068305435 ± 10000.806382 ± 5200.93
I & II66066005966 ± 10380.907162 ± 5351.09
I & II90073006390 ± 10640.887865 ± 5631.08
d-0.91-0.94-
RMSE (kg ha−1)-1065 872-
Table 4. Comparison of observed (Obs) and simulated (Sim) rice yield for the Wells cultivar from three versions of ORYZA from field data at Stuttgart, Arkansas, under varying planting dates from 2012 to 2015. Calibration (Cal) results for 2013 were shown (with values in Table S2) along with evaluation (Val) results for 2012, 2014, and 2015. The ratio of simulated to observed yield (Sim/Obs) was indicated along with Wilmott’s index of agreement and RMSE. Average maximum daily air temperature during flowering periods was shown as Tmax.
Table 4. Comparison of observed (Obs) and simulated (Sim) rice yield for the Wells cultivar from three versions of ORYZA from field data at Stuttgart, Arkansas, under varying planting dates from 2012 to 2015. Calibration (Cal) results for 2013 were shown (with values in Table S2) along with evaluation (Val) results for 2012, 2014, and 2015. The ratio of simulated to observed yield (Sim/Obs) was indicated along with Wilmott’s index of agreement and RMSE. Average maximum daily air temperature during flowering periods was shown as Tmax.
Tmax (°C)Obs Yield (kg ha−1)ORYZA-V1ORYZA-V2ORYZA-V3
Sim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/Obs
Cal28-March201330.2938298301.0510,3631.1111,4951.23
16-April201330.610,28910,0020.9710,0020.9711,3001.10
30-May201334.2892888670.9989901.0110,2531.15
17-June201329.1660777541.1750010.7656620.86
Average31.0880291131.0585890.9696781.08
d -0.92-0.92-0.86-
RMSE (kg ha−1) -633-952-1426-
Val30-March201230.111,90384150.7110,6630.9011,8631.00
11-May201231.010,79466640.6277930.7288670.82
26-March201428.312,05589670.7489670.7499950.83
18-April201429.211,34988180.7888180.7810,0590.89
2-May201431.1943291900.9791900.9710,5001.11
21-May201433.3902892161.0292161.0210,5451.17
5-June201430.4670877811.1677811.1689531.33
18-June201427.2796966790.8466790.8477050.97
3-April201533.8802084321.0586361.0897771.22
21-April201534.6701168520.9879301.1390831.30
5-May201534.7857461440.7275310.8886501.01
19-May201532.3902877760.8681200.9092771.03
3-June201528.3736483681.1483681.1495541.30
Average31.2917279460.8984380.9496021.07
d -0.53-0.66-0.66-
RMSE (kg ha−1) -2089-1606-1530-
Table 5. Comparison of observed (Obs) and simulated (Sim) rice yield for the CLXL753 cultivar from three versions of ORYZA from field data at Stuttgart, Arkansas under varying planting dates from 2012 to 2015. Calibration (Cal) results for 2013 were shown (with values in Table S2) along with evaluation (Val) results for 2012, 2014, and 2015. The ratio of simulated to observed yield (Sim/Obs) was indicated along with Wilmott’s index of agreement and RMSE. Average maximum daily air temperature during flowering periods was shown as Tmax.
Table 5. Comparison of observed (Obs) and simulated (Sim) rice yield for the CLXL753 cultivar from three versions of ORYZA from field data at Stuttgart, Arkansas under varying planting dates from 2012 to 2015. Calibration (Cal) results for 2013 were shown (with values in Table S2) along with evaluation (Val) results for 2012, 2014, and 2015. The ratio of simulated to observed yield (Sim/Obs) was indicated along with Wilmott’s index of agreement and RMSE. Average maximum daily air temperature during flowering periods was shown as Tmax.
Tmax (°C)Obs Yield (kg ha−1)ORYZA-V1ORYZA-V2ORYZA-V3
Sim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/ObsSim (kg ha−1)Sim/Obs
Cal28-March201332.611,95411,9871.0012,1641.0213,2871.11
16-April201330.012,00411,9691.0011,9691.0012,7731.06
30-May201333.810,54210,6061.0110,6061.0111,3701.08
17-June201331.5918094421.0383200.9194271.03
Average32.010,92011,0011.0110,7650.9811,7141.07
d -0.99-0.97-0.90-
RMSE (kg ha−1) -137-444-883-
Val30-March201231.313,82010,2650.7411,8490.8612,3300.89
11-May201234.012,66081460.6498460.7811,2010.88
26-March201427.611,14710,9360.9810,9360.9812,1881.09
18-April201430.212,76110,7480.8410,7480.8412,2520.96
2-May201431.012,10510,4350.8610,4350.8611,4030.94
21-May201433.312,10510,2030.8410,2030.8410,9600.91
5-June201430.6953392850.9792850.9710,0821.06
18-June201427.6907981420.9081420.9093951.03
3-April201532.511,34910,2830.9110,5240.9311,7421.03
21-April201534.611,44983650.7396580.8411,0820.97
5-May201534.711,70274850.6491820.7810,5350.90
19-May201532.911,34994860.8498960.8711,3161.00
3-June201528.3963410,2001.0610,2001.0611,1501.16
Average31.211,43895370.8310,0690.8811,2030.98
d -0.18-0.54-0.77-
RMSE (kg ha−1) -2429-1662-955-
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Li, S.; Fleisher, D.H.; Timlin, D.; Barnaby, J.; Sun, W.; Wang, Z.; Reddy, V.R. Improving Simulations of Rice in Response to Temperature and CO2. Agronomy 2022, 12, 2927. https://doi.org/10.3390/agronomy12122927

AMA Style

Li S, Fleisher DH, Timlin D, Barnaby J, Sun W, Wang Z, Reddy VR. Improving Simulations of Rice in Response to Temperature and CO2. Agronomy. 2022; 12(12):2927. https://doi.org/10.3390/agronomy12122927

Chicago/Turabian Style

Li, Sanai, David H. Fleisher, Dennis Timlin, Jinyoung Barnaby, Wenguang Sun, Zhuangji Wang, and V. R. Reddy. 2022. "Improving Simulations of Rice in Response to Temperature and CO2" Agronomy 12, no. 12: 2927. https://doi.org/10.3390/agronomy12122927

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop