Evaluation of Five Reanalysis Products over France: Implications for Agro-Climatic Studies

Abstract

Agro-climatic indicators (AgcIs) provide a suitable tool to evaluate the implications of climate change on agriculture by simplifying plant–climate interactions. However, developing reliable AgcIs requires high-quality historical climate datasets. Consequently, reanalysis products (RPs) are frequently used as a potential reference dataset for observed climate in agricultural studies. This study aims to compare five RPs (ERA5, ERA5-Land, SCOPE Climate, FYRE Climate, and RFHR) at reproducing observed AgcIs over France. The RPs are evaluated against the SYNOP meteorological data over the 1996–2012 period, focusing on six AgcIs specific to apple, maize, and vine crops. The findings show that RPs perform well in reproducing temperature-based AgcIs, with some slight discrepancies in areas with complex topography. However, all RPs tend to overestimate precipitation amounts and to underestimate dry days, leading to a poor performance in reproducing precipitation-based AgcIs. This study emphasizes the need for a thorough evaluation of the RPs in developing both temperature-based and precipitation-based AgcIs, especially if findings are intended to support operational agricultural decision-making.

1. Introduction

The agricultural sector is extremely vulnerable to climate change due to its sensitivity to meteorological parameters and its direct dependence on natural resources [1]. The altered precipitation patterns, warmer temperatures, and the increasing frequency of extreme weather events, including heatwaves, droughts, and floods, are already impacting crops worldwide. This not only affects yields and farming practices but also disrupts phenological events and compromises the overall quality of crops [2,3,4,5]. With a growing population, this raises urgent food security concerns at both local and global scales [6].

The assessment of how the current (and future) climate is suitable for agriculture is essential for many users in the agricultural community (planners, land managers, farmers, plant breeders, etc.). For this purpose, Agro-climatic Indicators (AgcIs) have been developed [7,8,9,10,11,12]. Derived from climate variables, AgcIs characterize plant–climate interactions, such as the impact of temperature on plant growth and productivity, and the response of plants to water stress and drought conditions. They provide synthetic information regarding climate influence on crop functioning. Over the last years, AgcIs were extensively used to better understand and manage climate-related risks in agriculture [7,8,9,10]. In addition to being understandable to the agricultural users, AgcIs are effective in encouraging stakeholders to consider the potential impacts of climate change on their land-use systems and in adapting to this issue [13].

High-quality historical weather datasets are a crucial component for developing reliable AgcIs. This implies having access to long observational meteorological variables with appropriate spatiotemporal resolution to describe local climate variability [14]. However, access to such high-quality historical weather datasets is often challenging due to the issues of missing data, temporal coverage, and spatial extent. Therefore, climate reanalysis products (RPs) are particularly attractive as they provide a comprehensive description of the observed climate as it has evolved during recent decades [15]. They combine past observations with forecast models to generate consistent time series of climate variables. Climate RPs are used in a wide range of applications including climate research, environmental monitoring and sectoral impacts assessments [16,17,18,19,20].

Even though the climate RPs provide the most complete numerical description of the climate, they inevitably contain some systematic biases (errors) from observations and imperfect forecast models, thus affecting their temporal and spatial consistency [21]. In addition, the data assimilation schemes used to adjust the weather model integration to actual observations is an additional source of uncertainty in the climate RPs. Many studies have evaluated and compared different climate RPs with observations in different parts of the world, at the global [22,23], continental [24,25,26,27], and regional [28,29,30,31] scale. Such studies have been conducted for traditional climate parameters, such as temperature and precipitation [28,29,30,31] and for extreme weather events [22,23,24,26] from a single or a few climate RPs for climate and hydrological applications. The findings of these studies are mixed, as differences in study regions, variables, and time scales result in inconsistency in the evaluation of the climate RPs performance, thus implying that a site-specific analysis is required. To date, few studies have evaluated the suitability of climate RPs for agricultural applications [32,33,34], including for the development of specific crops-related AgcIs [11,12]. However, to our knowledge, there is a notable gap in the literature associated with the evaluation of the accuracy of multiple climate RPs in the development of specific AgcIs for French crops.

The aim of this study is to evaluate five climate RPs with respect to observations, and to assess their reliability for constructing specific AgcIs to French crops. For this, five RPs (ERA5, ERA5-Land, SCOPE Climate, FYRE Climate, and the gridded dataset RFHR) are compared to multiple synoptic meteorological stations over the 1996–2012 period. The evaluation is made for six AgcIs, which encompasses both precipitation- and temperature-related indicators for three leading crops grown in France (apple, maize, and vine). Some implications for the agro-climatic studies are brought regarding the reliability of climate RPs at providing robust local AgcIs, which is a critical aspect that needs to be considered in the agriculture sector.

This paper is organized as follows. Section 2, presents the experimental design, including the study crops, the datasets, the AgcIs and the method used for the evaluation. Section 3 presents the relevant results of the evaluation of different climate RPs-AgcIs. The reliability of the climate RPs in constructing the AgcIs and the implications for agro-climatic studies are discussed in Section 4. Section 5 provides the concluding remarks.

2. Materials and Methods

2.1. Study Area

With around 812,000 hectares devoted to vines, France is the world’s second-largest wine producer and third-largest exporter in 2022 [35]. The main threats to vine include increasing temperatures and repeated droughts, resulting in premature grape ripening and altering the balance of sugars and acids. Extreme weather events, such as frost and hail, are also becoming major concerns, often leading to significant harvest damage or complete loss.

France is the leading producer of grain maize and the second largest producer of silage maize in Europe [36]. Maize is a summer crop; its growth and development are primarily influenced by temperature. In France, maize is the main irrigated crop, representing around one third of irrigated areas [37]. This is due to maize’s high sensitivity to drought and water deficit, particularly during the stages of reproductive development.

France is Europe’s third largest producer of apples [38]. Apple cultivation covers approximately 40,000 hectares, making it the dominant fruit species ($22\%$ of orchard surfaces) [37]. Typically cultivated in temperate climates, apple trees adapt to different temperature ranges, requiring a sufficient cold period and well-distributed rainfall throughout the growing season for optimum growth.

In this study, a total of 22 sites (10 for each crop, where 1 site can represent up to 2 crops) were selected, providing broad coverage, including varying climates and representing the main agricultural regions for each crop (Figure 1,

Table A1. Station name, coordinate and elevation of the weather stations used in this study.

Location Name	Longitude	Latitude	Elevation (m)
Abbeville	50.136	1.834	69
Caen-Carpiquet	49.18	$-0.456167$	67
Rouen-Boos	49.383	1.181667	151
Brest-Guipavas	48.444167	$-4.412$	94
Rennes-ST Jacques	48.068833	$-1.734$	36
Strasbourg-Entzheim	48.5495	7.640333	150
Nantess-Bouguenais	47.15	$-1.608833$	26
Tours	47.4445	0.727333	108
Bourges	47.059167	2.359833	161
Dijon-Longvic	47.267833	5.088333	219
Poitiers-Biard	46.593833	0.314333	123
Limoges-Bellegarde	45.861167	1.175	402
Clermont-FD	45.786833	3.149333	331
Bordeaux-Merignac	44.830667	$-0.691333$	47
Montelimar	44.581167	4.733	73
Embrun	44.565667	6.502333	871
Mont-De-Marsan	43.909833	$-0.500167$	59
Tarbes-Ossun	43.188	0	360
Toulouse-Blagnac	43.621	1.378833	151
Montpellier	43.577	3.963167	2
Perpignan	42.737167	2.872833	42

).

2.2. Data Sources

2.2.1. Observational Data

The observational meteorological data comes from the SYNOP network, provided by Météo France [39]. This consists of temperature and precipitation time series, recorded at 3-h intervals from January 1996 to present. The SYNOP data is considered as the reference dataset for the purpose of this study.

2.2.2. Reanalysis Datasets

The following section provides an overview of the five climate RPs used in this study, which are summarized in

Table 1. Details of the climate reanalysis products (RPs) datasets used in this study.

Dataset	Resolution		Coverage
	Spatial	Temporal	Spatial	Temporal
ERA5	${0.25}^{\circ }\times {0.25}^{\circ }$	Hourly	Global	1950–Present
ERA5-Land	${0.1}^{\circ }\times {0.1}^{\circ }$	Hourly	Global	1950–Present
SCOPE Climate	${0.08}^{\circ }\times {0.08}^{\circ }$	Daily	France	1871–2012
FYRE Climate	${0.08}^{\circ }\times {0.08}^{\circ }$	Daily	France	1871–2012
RFHR	${0.01}^{\circ }\times {0.01}^{\circ }$	Daily	France	1979–2021

.

ERA5 is the fifth generation of the ECMWF’s atmospheric reanalysis dataset [40]. It provides a comprehensive record of the global atmosphere, land surface, and ocean waves since 1950. ERA5-Land is an improved extension of the land component of ERA5 with an enhanced spatial resolution of around 9 km [41].

SCOPE Climate is a 25-member ensemble of 142-year daily high-resolution reconstructions of precipitation, mean temperature, and evapotranspiration over France. It results from the statistical downscaling of the global extended reanalysis 20CR [42], by the SCOPE method [43]. FYRE Climate results from the hybridization of the reanalysis FYRE Daily and FYRE Yearly, which results from assimilating historical station observations of temperature and precipitation into SCOPE Climate reconstructions [44].

RFHR (Reanalysis France High Resolution) is a high resolution (daily and 1 km) advanced climate reanalysis over France. It is built upon the ERA5-Land dataset to provide more accurate climate estimates available from 1979 to 2021. It results from a statistical downscaling followed by a bias correction according to the CDF-t method [45] using the ERA5-Land dataset and observed data available since 2012 on a 1 km grid covering all of France. The observed data are obtained from Météo-France’s radar network PANTHERE [46] for precipitation and Météo-France’s numerical weather forecast model AROME [47] for temperature. They are then reconstructed to achieve a spatial resolution of 0.01° and an hourly temporal resolution.

2.3. Agro-Climatic Indicators

To represent the potential effects of weather conditions on crop productivity and management, a set of six AgcIs were chosen based on the two common climate variables among the datasets: daily mean temperature and daily total precipitation. The AgcIs, summarized in

Table 2. Details of the agro-climatic indicators (AgcIs) used in this study.

Crop	Indicator	Equation
Vine	Winkler Index (WI, °C)	${\displaystyle \sum _{01/04}^{31/10}max\left(T_d-10,0\right)}$
	Hydrothermic index of Branas, Bernon, and Levadoux (HyI, mm °C)	${\displaystyle \sum _{01/04}^{31/08}{T}_m\times P_m}$
Maize	Flowering date (FD, day of the year)	${\displaystyle \sum _{15/04}min\left(max\left(T_d-6,0\right),30\right)}$ Date on which the threshold of 850 °C is reached
	Dry days (DDs, number of days)	${\displaystyle \sum _{01/06}^{31/08}{1}_{P_d<1}}$
Apple	Dormancy breaking (DB, day of the year)	${\displaystyle \sum _{30/10}CU\left(T_d\right)}$ $CU\left(T\right)=\left\{\begin{array}{cc}1-\left(\frac{\left|T-T_c\right|}{I_c}\right)\hfill & \mathrm{if}{T}_c-I_c<T<T_c+I_c\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.$ $T_c=0$ and $Ic=20$. Date on which the threshold of 56 CU is used
	Water requirements (WR, mm)	${\displaystyle \sum _{01/06}^{31/08}{P}_d}$

$P_d$ and $P_m$ are respectively daily and monthly total precipitations (mm). $T_d$ and $T_m$ are respectively daily and monthly mean temperature (°C).

, were selected to cover various aspects of climate impact on agriculture.

The concept of growing degree days (GDD) is first introduced before briefly describing the AgcIs below. GDD measures heat accumulation, assuming that plant and insect development largely depends on heat exposure during the growing season, rather than calendar days. It is calculated as the accumulated sum of the difference between daily mean temperature and a species-specific base temperature, below which growth is limited.

• Winkler Index (WI), also known as the Winkler Scale or Winkler Regions, is commonly applied to classify the climate of wine growing regions based on daily temperature converted to GDD during the growing season (1 April to 31 October in the Northern Hemisphere) [48]. The regions are classified into five climate regions, characterized by increasing temperature levels: from Region I (<1390 °C, for example Champagne) to Region V (>2220 °C, for example Palermo);
• Hydrothermic index of Branas, Bernon, and Levadoux (HyI) developed by [49] considers both precipitation and temperature regimes to estimate the risk of downy mildew for grapevines. An HyI value below 2500 °C mm represents a low risk of infection, whereas a value above 5100 °C mm is considered as a high risk of infection;
• Flowering date (FD) for maize, also known as R1 or silking stage, begins when female flowers (i.e., silks), first appear outside the husk leaves [50]. It is reached when at least $50\%$ of a field’s plants show visible silks. This stage initiates the reproductive phase and is crucial to monitor as it determines the harvest date and therefore its quality and yield. It can be estimated using a modified GDD method, using a 6 °C base temperature, with an upper temperature threshold of 30 °C, starting from the date of planting;
• Dry days (DDs) Maize is a water-demanding crop and its water requirements vary by growth stage [51]. Peak demand occurs in early reproductive stages; around the pre- and post-flowering stage (June to August). During this period, water stress should be avoided as the drought negatively impacts plant and grain development, ultimately reducing yield [52]. One way to measure dry conditions is by counting dry days;
• Dormancy breaking (DB) To break dormancy and initiate flowering, most fruit trees need to accumulate a certain amount of cold [53], referred to as chilling requirement (CR) and measured in chilling units (CU). For apple trees, cold needs are satisfied by daily mean temperatures between $-20$ °C and $+20$ °C, with temperatures close 0 °C being the most effective. To estimate CU, various models are available. Here, a triangular function inspired by the F1GOLD1 model is used, which has been previously applied to the “Golden Delicious” apple [54]. CR is then satisfied when the cumulative CU, starting from 30 October, reaches a specific threshold;
• Water requirements (WR) Apple trees have a high-water content and require more water during the fruit production season, approximately June through August. Finding a balance between rainfall and crop water demand during this time is necessary, not only to help conserve water but also to avoid yield losses and to produce high-quality fruits [55].

2.4. Comparison Approach

To ensure a consistent comparison, the spatial coverage of climate RPs is matched to that of the reference data by extracting grid values at the coordinates of the selected SYNOP stations. For each station location, climate RP and, if necessary, ensemble members, daily time series of precipitation and mean temperature is calculated using bilinear interpolation of the four nearest neighbors (grid cells). In cases where bilinear interpolation is not suitable within the gridded land dataset, nearest neighbor interpolation is used. The AgcIs were then calculated for each year using the interpolated data. The period of comparison is 1 January 1996 to 29 December 2012; this 17-year period is common among all the datasets.

In the following, $X_{rp}=\left(x_1^{rp},\dots ,x_n^{rp}\right)$ will represent RP-derived values and $X_{ref}=\left(x_1^{ref},\dots ,x_n^{ref}\right)$ will represent the observed values from the reference dataset. The performance of the climate RPs was evaluated based on three metrics:

• Lin’s concordance correlation coefficient (LCCC), first introduced by [56], quantifies the agreement between two continuous measures made by different observers/methods of the same variable by measuring the variation of their linear relationship from the ${45}^{\circ }$ line through the origin. The degree of concordance between $X=X_{ref}$ and $Y=X_{rp}$ is estimated using:

  $${\widehat{\rho }}_c={\displaystyle \frac{2S_{XY}}{S_X^2+S_Y^2+{\left(\overline{X}-\overline{Y}\right)}^2}},$$

  where ${\displaystyle \overline{X}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{x}_i, S_X^2={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(x_i-\overline{X}\right)}^2,}$ and ${\displaystyle S_{XY}={\displaystyle \frac{1}{n}}\sum _{i=1}^n\left(x_i-\overline{X}\right)\left(y_i-\overline{Y}\right)}$.
  $\widehat{\rho }$ will represent the degree of agreement between the climate RP and the reference dataset, with values ranging from $-1$ to 1. A value of 1 indicates perfect agreement, 0 indicates no agreement, and −1 indicates perfect disagreement.
• Percent Bias (PBias) quantifies the average tendency of the RP-derived values to be larger or smaller compared to the observed values.

  $$PBias=100\times {\displaystyle \frac{1}{n}}\sum _{i=1}^n\left({\displaystyle \frac{x_i^{rp}-x_i^{ref}}{x_i^{ref}}}\right).$$
  A value close to zero indicates that the AgcI have similar average values, a positive value means that the evaluated product tends to overestimate the reference values, and a negative bias shows an underestimation against the reference values.
• The integrated quadratic distance (IQD), introduced by [57], is defined as the integral over the squared difference between the corresponding cumulative distribution functions (CDFs).

  $$IQD=d\left(F_{rp},F_{ref}\right)={\int }_{-\infty }^{+\infty }{\left(F_{rp}\left(x\right)-F_{ref}\left(x\right)\right)}^{2}dx.$$
  Specifically, d is a distance that compares the corresponding empirical cumulative distributions $F_{rp}$ and $F_{ref}$ and returns a positive numeric value summarizing their differences with $IQD=0$ if $F_{rp}=F_{ref}$. More generally, a lower value indicates a smaller difference between $F_{rp}$ and $F_{ref}$.

3. Results

The AgcIs derived from the climate RPs are compared to those calculated with the reference dataset over the 1996–2012 period using the three metrics (PBias, IQD and LCCC). This assessment was conducted by considering all stations together (

Table 3. Metrics between the reference dataset and RPs over the 1996–2012 period across all study sites for each AgcI.

Metric	Dataset	Temperature-Based AgcIs			Precipitation-Based AgcIs		Mixed AgcIs
		WI	FD	BD	DDs	WR	HyI
LCCC	ERA5	0.97	0.84	0.9	0.45	0.75	0.67
	ERA5-Land	0.95	0.82	0.94	0.49	0.75	0.64
	SCOPE Climate	0.95	0.91	0.94	0.61	0.58	0.57
	FYRE Climate	0.99	0.97	0.97	0.9	0.94	0.92
	RFHR	0.98	0.96	0.98	0.71	0.69	0.61
PBias	ERA5	$-1.71$	0.46	14.61	$-16.59$	22.23	19.35
	ERA5-Land	$-3.08$	1.36	6.27	$-15.51$	22.35	18.98
	SCOPE Climate	$-4.07$	1.2	4.871	$-6.57$	9.74	1.9
	FYRE Climate	2.33	$-0.6$	$-0.98$	$-2.95$	4.55	3.5
	RFHR	0.63	0.21	$-3.37$	$-7.43$	12.69	21.11
IQD	ERA5	2.1	0.14	0.12	3.64	0.78	88.45
	ERA5-Land	3.55	0.36	0.02	3.34	0.69	75.63
	SCOPE Climate	5.09	0.18	0.13	0.8	0.64	15.52
	FYRE Climate	2.04	0.05	0.06	0.16	0.08	8.86
	RFHR	1.16	0.02	0.03	0.99	0.1	80.24

, referred to as the ’global scale evaluation’) and by taking each station individually (referred to as the ’local scale evaluation’). For FYRE Climate and SCOPE Climate, which are both a 25-member ensemble, the metrics are calculated for each member independently and the performance is evaluated on the ensemble mean.

3.1. Evaluating the RPs for Vine Crop Indicators

On a global scale, all RPs show a high agreement with the reference dataset in reproducing WI, with LCCC values greater than $0.95$ (Table 3). For most of the stations ($40\%$, $70\%$, $80\%$, $90\%$, and $90\%$ for SCOPE Climate, ERA5-Land, ERA5, FYRE Climate, and RFHR, respectively), RPs exhibit high correlations (LCCC values $⩾0.8$), showing their reliability at reproducing WI (Figure 2d). RFHR has the lowest IQD value, indicating a closer match with the reference dataset, followed by FYRE Climate and ERA5, which exhibit similar levels of agreement. Meanwhile ERA5-Land and SCOPE Climate show relatively higher dissimilarity (Table 3). The spatial distribution of IQD values reveals varying levels of agreement with the reference dataset across the study sites (Figure 2f). Furthermore, as shown in Table 3 and Figure 2b, FYRE Climate and RFHR slightly overestimate WI values (the kernel density estimation (KDE) curve is right-shifted), while the other RPs tend to underestimate WI values. Figure 2e further confirms that the performance of the RPs varies across the study sites.

Overall, RPs are able to reproduce the observed WI values. However, some discrepancies are seen at the Bordeaux-Mérignac station, where SCOPE Climate shows poor performances, and at Montelimar, where a significant dissimilarity compared to the reference dataset is seen for ERA5-Land.

According to Table 3 and Figure 3d, FYRE Climate shows strong similarity with the reference dataset when reproducing HyI values; FYRE Climate consistently shows the highest level of agreement, when considering all study sites collectively (with LCCC = $0.92$) and for each individual location (LCCCs $⩾0.8$). ERA5, ERA5-Land, RFHR, and SCOPE Climate, however, have weaker correlations with the reference dataset, with SCOPE Climate displaying the lowest agreement with the observations. IQD values affirm the earlier finding (Table 3 and Figure 3f); FYRE Climate consistently displays superior agreement with the reference dataset for when reproducing HyI values. However, SCOPE Climate demonstrates a closer match with the reference dataset, in terms of IQD, compared to ERA5, ERA5-Land, and RFHR. The three latter exhibit similar results, suggesting a comparable level of agreement with the reference dataset. All RPs tend to overestimate the HyI values (Table 3 and Figure 3e), with FYRE Climate and SCOPE Climate showing a low degree of overestimation compared to the other RPs. However, a significant variability in the bias values is seen which suggests a different level of agreement between the RPs and the reference dataset across the study period and sites.

Overall, the results highlight the consistent outperformance of FYRE Climate compared to other RPs; a similar performance between ERA5 and ERA5-Land is seen. RFHR shows an overall similar performance compared to ERA5-Land, with an improvement in reproducing HyI values for $60\%$ of the stations.

3.2. Evaluating the RPs for Maize Crop Indicators

The correlations between observed and RP-derived values of FD are relatively strong for all products, with RFHR and FYRE Climate showing the highest correlations with the reference dataset (Table 3). Good agreement between RPs and the reference dataset is also seen for the study sites, with LCCCs greater than $0.7$ for $86\%$ of cases (considering all stations and all RPs) (Figure 4d). IQD values further confirm the previous results. A comparable agreement between the RPs and the reference dataset is seen on both global and local scales (Table 3 and Figure 4c,f). Overall, RPs display minimal biases (Figure 4b). Figure 4e illustrates that, on a local scale, the bias values remain insignificant, with values ranging between $-2\%$ and $2\%$, in most cases (considering all stations and RPs).

Overall, the results show a good level of precision in reproducing the FD values by the RPs, with no significant overestimation or underestimation. None of the RPs stands out as best for all stations; the most notable differences are seen for the Clermont-FD and Tarbes-Ossun stations, where ERA5 and ERA5-Land perform rather poorly at reproducing the FD values compared to the other RPs.

Regarding the DDs indicator, FYRE Climate displays the highest correlation with the reference dataset with a global coefficient of 0.9 and coefficients greater than $0.8$ for $90\%$ of stations (Table 3 and Figure 5d). The rest of the RPs exhibit relatively weaker correlations, with RFHR performing slightly better in reproducing DDs values. LCCC values for RFHR remain quite acceptable, with a global value of $0.71$ and correlations greater than $0.6$ for $70\%$ of the stations—a notable improvement compared to ERA5, ERA5-Land, and SCOPE Climate, where only $10\%$ of stations exhibit similar correlations. The same findings are shown by IQD values (Figure 5c,f). FYRE Climate is the closest to the reference dataset, with a global IQD of $0.16$, followed by RFHR and SCOPE Climate. ERA5 and ERA5-Land show poor performances with an IQD greater than 3 in most cases.

For each RP, the CDF curve is consistently and systematically shifted downward (i.e., towards the lower values) of the reference line, indicating an underestimation of DDs values (Figure 5c). Results in (Table 3, Figure 5b,e) further confirm this assumption, where bias values are always negative. Although RFHR still underestimates the DDs indicator, a slight improvement is seen compared to ERA5-Land at reproducing the DDs values; the KDE curve is shifted toward the right (zero) and the average bias is reduced by approximately half.

3.3. Evaluating the RPs for Apple Crop Indicators

For DB indicator, strong correlations between all RPs and the reference dataset are observed (LCCC values $⩾0.9$, Table 3). For most stations, the LCCC values between the RPs and the reference dataset are high. The ratio of stations with LCCC values greater than $0.9$ is $30\%$, $70\%$, $80\%$, $80\%$, and $100\%$ for SCOPE Climate, ERA5, FYRE Climate, ERA5-Land, and, RFHR respectively (Figure 6d). Regarding the PBias, RPs exhibit, on average across all sites, values ranging between $-3\%$ and $15\%$ (Table 3). Compared to RFHR, the KDE curves of the other RPs show a large spread, suggesting a higher variability (Figure 6b). For example, ERA5 overestimates the DB date by more than $370\%$. Figure 6c shows good agreement between the cumulative probability distributions of RPs and the reference dataset. The IQD values also suggest a similarity between the RPs and the reference dataset, with values below $0.3$ in most cases (considering all stations and RPs) (Figure 6f).

While RPs lead to satisfactory and similar results for most of the stations, the Embrun site presents an exception. All RPs perform rather poorly for this station except for RFHR, which returns satisfactory results (LCCC = $0.9$, PBias = $3.77$, and IQD = $0.07$).

As illustrated in Table 3 and Figure 7d, FYRE Climate displays the best agreement with the reference dataset for the WR indicator. Despite the lower LCCCs, the agreement between ERA5, ERA5-Land, RFHR, and the reference dataset can be considered as acceptable. In contrast, SCOPE Climate consistently shows a lower agreement with the observations (on a global scale and for $80\%$ of stations), with LCCCs below $0.6$. Regarding the IQD, FYRE Climate gives the lowest values (Table 3 and Figure 7c), indicating a closer similarity to the reference dataset compared to the other RPs. ERA5 and ERA5-Land have relatively close values, indicating comparable performance. Despite the lower IQD value, RFHR still displays higher values compared to ERA5 and ERA5-Land for $60\%$ of the stations (Figure 7f). This discrepancy between RFHR and the reference dataset is particularly notable at the Brest-Guipavas station, where the IQD value is three times higher compared to ERA5 and ERA5-Land. In addition, all RPs exhibit positive biases on average, suggesting a tendency towards an overestimation of WR (Table 3). Except for the Brest-Guipavas station, for which all the RPs show an underestimation of WR compared to the reference dataset, the overestimation of WR is still noticeable in most cases (considering all the stations and RPs) (Figure 7e).

4. Discussion

Crop modeling studies rely on various input factors, such as soil characteristics, crop management practices, and climate data to simulate crop growth, development, and yield. However, one of the major limitations to the application of such studies is the absence of accurate observational climate data with high temporal and spatial resolutions. Even in data-rich countries like France, limitations persist due to issues such as the heterogeneity of observation station networks, incomplete spatial coverage, and uncertainties in measurements. One way to overcome these challenges is by using climate reanalysis datasets [3,16,17]. However, it is important to note that climate RPs have some limitations. These include errors in numerical models, uncertainties introduced during data assimilation, and homogeneity issues due to changes in observing systems.

The ability of RPs to reproduce observed AgcIs has been investigated here. Five RPs were compared based on a set of six AgcIs specific to vine, maize and apple. The study focuses on 22 synoptic meteorological stations in France over the 1996–2012 period.

4.1. Reliability of the Climate RPs for Developing AgcIs

The findings reveal a satisfactory degree of agreement among the climate RPs in reproducing temperature-based indicators. The five RPs considered in this study exhibit a satisfactory level of consistency with the observations with significant correlations (LCCC) and low differences (PBias). Overall, FYRE Climate and RFHR show the best performances at reproducing all temperature-based indicators, while SCOPE Climate consistently exhibits the lowest performance. Nonetheless, it is worth noting that some discrepancies among the RPs become evident when reproducing temperature-based indicators in sites with challenging topography (e.g., high altitudes or complex terrain), such as for the Embrun or Clermont-FD stations. High-resolution RPs, such as RFHR, provide the best results for the temperature-based indicators for specific mountainous stations, highlighting their advantage for a more detailed representation of complex terrain and microclimates. Similarly, various studies reported the difficulties of accurately representing temperature within coarse spatial resolution RPs in areas under a complex topography [25,31,58].

For precipitation-based indicators, the results show poorer performances compared to temperature-based indicators. RPs exhibit a lower level of agreement with the reference dataset and more pronounced discrepancies among themselves. Several studies have reported similar findings [22,28], highlighting the RPs’ challenges in accurately representing precipitation-based indices, regardless of their temporal scale. Additionally, the results consistently demonstrate RPs’ tendency to overestimate the indicator linked to precipitation amount WR while simultaneously underestimating the one associated with precipitation occurrence DDs. This dual effect within the RPs can be primarily attributed to the presence of a wet bias coupled with a tendency to underestimate the frequency of dry days. This behavior is supported by multiple studies conducted in various regions of Europe [27,29,30].

Regarding the mixed indicator (HyI), the findings reveal similar behaviors among the RPs to that observed when reproducing precipitation-based indicators. All the scores are lower compared to those of the temperature-based indicators. Additionally, a consistent pattern of overestimating HyI values by the RPs, resembling the case of WR, is also seen. While the importance of temperature is unquestionable here, it appears that precipitation has a more dominant influence on the performance of RPs in reproducing HyI.

Overall, FYRE Climate provides the best reproduction of the precipitation-based indicators. RFHR, ERA5, and ERA5-Land display similar performances in reproducing AgcIs based on precipitation amount, and a slight improvement is observed with RFHR’s for the representation of precipitation occurrence. Despite the well-marked accuracy of FYRE Climate in representing AgcIs, some limitations are noteworthy. One key constraint is the data’s temporal extent until 2012, which constrain the climate simulations beyond 2012. Additionally, only temperature and precipitation data are provided. Even though these two variables are important, they do not capture the full complexities of agro-climatic interactions.

Considering these findings, further studies should extend their investigation to the RPs reliability in reproducing AgcIs, considering a larger set of climatic variables beyond temperature and precipitation. While these two variables are fundamental, they do not fully capture the complexities of agro-climatic conditions. This expansion is essential since various climatic factors, such as humidity, wind, and solar radiation, significantly influence agricultural outcomes.

4.2. Implications for Agro-Climatic Studies

Accurate reproduction of temperature-based AgcIs through RPs provides valuable insights into the complex interactions between temperature and crop. This enhances the ability of assessing the effects of temperature on crops, soil conditions, and overall agricultural productivity. This is crucial for providing informed decision-making, such as planting schedules, crop selection, irrigation, and other agricultural practices. In addition, this will improve the modeling and forecasting of plant growth, disease development, and pest dynamics, and will ultimately lead to improved efficiency and yields of crops.

An accurate reproduction of AgcIs based, solely or partly, on RPs precipitation is challenging due to the dynamic and stochastic nature of precipitation, varying in intensity and frequency. The uncertainties in such precipitation-related AgcIs can influence the assessments of agricultural productivity, water management, and decision-making. For instance, underestimating the frequency of dry days could compromise the accurate characterization of dry/wet spell lengths and drought periods [11,26]. Additionally, overestimating precipitation amounts can result in an inaccurate evaluation of water availability, underestimation of irrigation requirements, or exaggeration of potential crop yields. This challenge extends to disease management decisions, as seen in the case of HyI, where such mismanagement can lead to unnecessary or excessive control measures, with potential costs and environmental consequences. All these outcomes can lead to ineffective irrigation scheduling and practices, inadequate drought preparedness, and, ultimately, negative impacts on crop yield and quality.

In this study the need for critical evaluation in agro-climatic assessments is highlighted. Understanding both the strengths and limitations of RPs holds significant implications for agro-climatic studies. In a changing climate, where the increase in temperatures and shifts in precipitation patterns are anticipated, precision in agriculture assessments is crucial. Relying on inaccuracies could not only amplify uncertainties in climate change assessments but also limit the ability to make the distinction between climate change impacts and data errors. Therefore, a comprehensive understanding of RPs and their limitations is crucial, along with validation against local observations, and cautious interpretation and application. All these elements are essential for informed decision-making in agriculture and climate change adaptation.

5. Conclusions

Climate RPs serve as valuable tools for climate research. Their ability to reproduce observed AgcIs has been investigated here. Five RPs were compared based on a set of six AgcIs specific to vine, maize, and apple. The study focuses on 22 synoptic meteorological stations in France over the 1996–2012 period. This study offers a comprehensive assessment of the reliability of climate RPs in reproducing some AgcIs. The results reveal significant variations among the different RPs when reproducing such indicators, with their performance exhibiting distinct patterns linked to the underlying climatic variables. RPs have proven their robustness in accurately replicating temperature-based indicators, despite some disparities in challenging terrains, especially for coarse-resolution products. However, the complexities surrounding the accurate representation of precipitation based AgcIs remain. These findings underscore the ongoing challenges in achieving precise representations of this crucial aspect in agro-climatic assessments. The study provides valuable insights into the strengths and limitations of climate reanalysis products (RPs), encouraging a more careful and context-sensitive application in agricultural research and planning.