Performance of the State-Of-The-Art Gridded Precipitation Products over Mountainous Terrain: A Regional Study over Austria

Abstract

During the last decade, satellite-based precipitation products have been believed to be a potential source for forcing inputs in hydro-meteorological and agricultural models, which are essential especially over the mountains area or in basins where ground gauges are generally sparse or nonexistent. This study comprehensively evaluates several newly released precipitation products, i.e., MSWEP-V2.2, IMERG-V05B, IMERG-V06A, IMERG-V05-RT, ERA5, and SM2RAIN-ASCAT, at daily and monthly time-scales over Austria. We show that all the examined products are able to reproduce the spatial precipitation distribution over the country. MSWEP, followed by IMERG-V05B and -V06A, show the strongest agreement with in situ observations and perform better than other products with respect to spatial patterns and statistical metrics. Both IMERG and ERA5 products seem to have systematic precipitation overestimation at the monthly time-scale. IMERG-V06A performs slightly better than IMERG-V05B. With respect to heavy precipitation (P > 10 mm/day), MSWEP compare to other products demonstrate advantages in detecting precipitation events with a higher spatial average of probability of detection (POD) and lower false alarm ratio (FAR) scores skill (0.74 and 0.28), while SM2RAIN and ERA5 reveal lower POD (0.35) and higher FAR (0.56) in this precipitation range in comparison with other products. However, the ERA5 and MSWEP indicate robust average POD and FAR values with respect to light/moderate precipitation (10 mm > P ≥ 0.1 mm) with 0.94 and 0.11, respectively. Such robustness of MSWEP may be rooted in applying the daily rain gauges in calibration processes. Moreover, although all products accurately map the spatial precipitation distribution they still have difficulty capturing the effects of topography on precipitation. The overall performance of the precipitation products was lower in the peripheries of the study area where most stations are situated in the mountainous area and was higher over the low altitude regions. However, according to our analysis of the considered products, MSWEP-V2.2, followed by IMERG-V06S and -V05B, are the most suitable for driving hydro-meteorological, agricultural, and other models over mountainous terrain.

1. Introduction

Droughts and floods are water-related natural phenomena which have large negative impacts on society and activities related to agriculture, and local economies. Drought is one of the most important natural disasters, since it affects wide areas for long time (months to years) and, thus, has a serious impact on regional or countries economic performance, etc.

In recent decades, large-scale extreme events (i.e., droughts) have been observed in many places around the world leading to high negative impacts on economic, ecological resources, food shortages, etc. However, floods are among the most destructive natural phenomena, declaring more lives and leading to more property damage than any other natural events. The reliable and accurate drought and flood information have been more interested for a variety of authorities, such as water managers, policy makers, researchers, farmers, etc., for effective management [1].

Since precipitation is the most important factor of the aforementioned phenomena, knowing the locations, domain, and length of precipitation is essential to understand, predict, and mitigate the impact of such disasters. Irrespective of the less accessible mountainous and oceanic regions, compared to ground-based measurements, such as gauges and radars, satellite precipitation estimates (SPE) products are able to cover the precipitation system at a nearly global-scale. Generally, in situ observations are often subject to wind effects, many missing values, insufficient number of stations, or sparse gauge networks, particularly in mountainous or desert areas [2,3]. Moreover, gridded daily surface precipitation data are important for many water-related applications, such as drought and flood monitoring systems. Rapid growth in computer technology and the remote sensing area help observations processed from satellites, individually, and merge them with other data sources to provide a better understanding of the precipitation spatial visualization. The information derived from SPEs provides tremendous potential for identification, monitoring, and assessment of droughts, flood, etc., especially for regions with sparse rain gauges or limited radar coverage [4].

However, the precision of SPEs at spatiotemporal representations has a great influence on the effective predictions of natural hazard, climate impacts, etc.; therefore, accuracy analysis of the new precipitation products is often applied before it can be employed in decision-making activities [5]. The satellite/gridded data produce the area average of precipitation in contrast with point measurements obtained with rain gauges. Earlier studies have shown that diverse altitude and geographic and climate conditions have greatly impact on the accuracy and performance of satellite or other precipitation products [2,5,6,7,8,9,10,11]. For instance, in a study by Gottschalck et al. [12] the overestimation of 3B42-RT over the Central United States is attributed to misclassification of cold cirrus clouds as precipitating systems. In another study, Dinku et al. [13] demonstrated that topography plays a significant role in SPE due to the weakness of algorithms to detect orographically-induced precipitation. Numerous studies have shown that an appropriate interpolation method might develop a gridded dataset using the rain gauges, but the obtained dataset is dependent on both adequate underlying station observations and the use of an appropriate interpolation technique to produce high-resolution gridded point estimates prior to the creation of area-averages grid values [14]. Thaler et al. [15] used different gridded precipitation data in an agronomic application and analyzed how different products influence a crop model application. The Austrian radar network (Austrocontrol) is operated consisting of four radar stations distributed across the country. However, due to the mountainous area and terrestrial characteristics of the country, in many western regions of Austria radar data have limitation to use particularly during wintertime, when precipitation may originate from rather shallow cloud systems.

In recent years, with the rapid development of remote sensing technique, more and more quasi-global satellite precipitation products have been produced and released to the public. The objective of this paper is inter-comparison of the recently released gridded precipitation products (e.g., MSWEP-V2.2, IMERG-V05B, IMERG-V05-RT -V06A, ERA5, and SM2RAIN) against in situ observations. The question motivating this study is “To what extent have the recently released gridded precipitation products, improved as compared to a dense rain-gauge network?” The total error is broken down into estimation and detection of precipitation in order to assess the algorithm performance than can highlight the weakness and strength of those algorithms and assist the developer to improve those aspects that have greater need.

In this study, the gauge-based measurements are directly compared against the corresponding gridded data, enabling us to identify the best precipitation estimated product and how close they are to source data (stations).

It is noteworthy to mention that, previously, Sharifi et al. [3] conducted a study to evaluate the reliability of the IMERG-V04 final-run (FR) and real-time (RT) products against the Central Institute for Meteorology and Geodynamics (ZAMG) stations over Northeast Austria. However, an evaluation of the performances of ERA5, MSWEP-V2.2, IMERG-V5B, IMERG-V6, IMERG-V5-RT, and SM2RAIN products has not been conducted over Austria. Therefore, the aforementioned products and a highly dense in situ precipitation network provided by the Federal Ministry for Sustainability and Tourism (BMNT)-Austria (882 stations) are selected in this study. Lastly, this study will be useful since it will provide the reference for precipitation monitoring and regional climate prediction across Austria.

2. Data and Study Area

The data used in this study are described below and its summaries and characteristics have been shown in

Table 1. Characteristics of the precipitation products.

Products	Temporal Resolution	Spatial Resolution	Coverage	Availability
IMERG-V05B	30 min	0.1°	60°N–60°S	March 2014–present
IMERG-V05-RT	30 min	0.1°	60°N–60°S	March 2014–present
IMERG-V06A	30 min	0.1°	60°N–60°S	June 2014–present
MSWEP-V2.2	3 h	0.1°	90°N–90°S	1979–2017
ERA5	1 h	0.28°	90°N–90°S	1950–present
SM2RAIN	Daily	12.5 km	90°N–90°S	2007–2018

.

2.1. In Situ Data

In this study, we use precipitation data from 782 stations provided by BMNT, followed by proper quality-control checks within Austria, in the range of daily time-scale between 1985 and 2016.

However, there are other stations provided by ZAMG-Austria distributed throughout the country. In this study the in situ observation network provided by BMNT (hereafter: eHYD) is used for two reasons: first, the eHYD network is much denser than the ZAMG stations. Second, eHYD provides gauge observations that are entirely independent of the gridded precipitation products that are assessed in this study, while a large number of ZAMG observations are used by the Global Precipitation Climatology Centre (GPCC) products, which, in turn, are integrated in several of the products assessed in this study (i.e., IMERG). As an example, Sharifi et al. [3] reported that over the northeast of the country 51 out of 62 ZAMG synoptic stations are shared with GPCC, which contains a major part of the total ZAMG gauges (82%). Since the satellite products used in this study are used with GPCC gauge data for calibration, we utilize the independent eHYD observations in order to avoid any misleading results. Spatial distribution of both eHYD and ZAMG stations and elevation map of Austria are shown in Figure 1.

It is worthy to mention that stratiform, convective, and orographic precipitation types occur over the country [16].

2.2. Precipitation Products

2.2.1. MSWEP V2.2

Multi-Source Weighted-Ensemble Precipitation (MSWEP) uses a combination of the gauge-, satellite-, and reanalysis-based data to provide a reliable precipitation estimates over the entire globe. With the consistent precipitation record from 1979 until the near present, it enables assessing the precipitation trend, drought, etc. In this study, the newest version of this product (V2.2) is used. The dataset cover the time period from 1979 to 2017 and is available at 0.1° spatial resolution and at three-hourly, daily, and monthly temporal resolutions [17]. The historic MSWEP data are freely available via www.gloh2o.org.

2.2.2. ERA5

The precipitation data obtained from ERA5 forecast product developed through the Copernicus Climate Change Service (C3S). This product has global spatial coverage and covering the period 1950 to the present. Precipitation data are available at an hourly resolution and consist of forecasts initialized twice daily from analyses at 06 and 18 UTC. The ERA5 high-resolution atmospheric data has a resolution of 0.28125 degrees (31 km) but, when downloading the data, there is a possibility to resampling the data to a higher spatial resolution. This will be done for continuous parameters by default through bilinear resampling methods [18]. Therefore, for consistency to other satellite data, we used the precipitation data with 0.1° spatial resolution. The data are freely accessible to users from Climate Data Store (CDS) website (https://cds.climate.copernicus.eu).

2.2.3. IMERG-FR (Final-Run) V05B

The Global Precipitation Measurement (GPM) mission’s Precipitation Processing System (PPS) at NASA’s Goddard Space Flight Center released IMERG-V05B data to the public in late November 2017. The dataset includes precipitation since March 2014. IMERG datasets are freely accessible to users from NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) website (https://disc.gsfc.nasa.gov). The IMERG-FR product is produced based on merging and interpolating all constellation microwave sensors and IR-based observations, and calibrated with monthly precipitation rain gauges from GPCC and create research-level products [19].

2.2.4. IMERG-RT (Real-Time) V05

The data are available from March 2014 to the present, thus providing IMERG-RT V03 data for the period 2014, to December 2015. The IMERG-RT late run, which is the near real-time product of the IMERG. It runs about ~10–14 h after the observation time and it is calibrated with the monthly climatological data (unlike the Final-Run, which uses monthly gauge data).

2.2.5. IMERG-FR (Final-Run) V06A

In the new version of IMERG, datasets from the Tropical Rainfall Measuring Mission (TRMM) have been used as a calibrator for GPM for the nearly time period of TRMM start its observation, to allow GPM processing spin up and a graceful transition from the TRMM era to the GPM era. This product was released on 26 March 2019 and the first available GPM-era products start with June 2014 until December 2016 and soon it will span 2000 until the present [20].

2.2.6. SM2RAIN-ASCAT

SM2RAIN obtained from ASCAT (Advanced SCATterometer) satellite soil moisture data [21]. The SM2RAIN-ASCAT rainfall dataset is provided over an irregular grid at 12.5 km on a global scale. The product represents the daily cumulated rainfall. The SM2RAIN method was applied to the ASCAT soil moisture product for the period from 2007 to 2018.

3. Methodology

For our study period, the 782 eHYD gauges were used in this domain. The eHYD accumulates daily precipitation ending at 07:00 o’clock local time, which is different from the satellite daily precipitation accumulation convention. Therefore, the sub-daily precipitation estimates (e.g., half-hourly, three-hourly) were aggregated and derived a meaningful daily (at the local time of gauge measurement) and, consequently, monthly data. Then a comparison among the eHYD gauge-based data and the gridded precipitation products across Austria was conducted. It should be mentioned since the SM2RAIN data are available only in daily (and not sub-daily) scale, therefore, we could use only the original SM2RAIN daily data which estimate rainfall between the 00:00 and the 23:59 UTC of the indicated day.

In this research, to evaluate the capability of precipitation products to capture the precipitation patterns and for the aim of intercomparison with other gridded precipitation products, the data of a pixel of the gridded products are compared with that corresponding to the ground point observation (i.e., the station). Only the cells where there is at least one reporting station can be selected for computation. Since we have used a dense network, in numerous pixels, there are two or more gauges are located. In this case, an average of the two or more gauges is used as the basis for comparison. At the end, all stations fell within 601 pixels over the country.

The assessment and validation of precipitation products are carried out based on continuous and categorical statistical metrics. To quantitatively compare the performance of the gridded products against in situ observations at daily and monthly time-scales, the continuous statistical metrics including correlation coefficient (CC), bias, mean absolute error (MAE), and root mean square error (RMSE) are used. The CC is used to assess the agreement between SPEs and rain-gauge observations. The CC value vary from −1 to +1, where +1 indicates a perfect skill score and −1 indicates a perfect negative linear correlation. The bias is defined as the average difference between in situ observations and satellite/model precipitation estimates, and can be either positive or negative. A negative bias indicates underestimation by satellite precipitation while a positive bias indicates overestimation. The MAE is used to represent the average magnitude of the error and MAE = 0 indicates a perfect score. The RMSE is used to measure the average error magnitude and weighs the errors according to their squared value. This gives a greater weight to larger errors than the MAE. RMSE = 0 represents a perfect score. To examine the capability of the products in detection of precipitation, the two categorical statistical metrics, probability of detection (POD) and false alarm ratio (FAR), are used (see Appendix A). POD is an indicator of the SPE’s ability to correctly detect precipitation events. Values vary from 0 to 1, with 1 as a perfect score. FAR denotes the fraction of cases in which the SPEs record precipitation when the rain gauges do not. Values vary from 0 to 1, with 0 as a perfect score. For extreme precipitation, we use R90th index to measure extreme wetter condition. R90 is precipitation in the 90th percentile of wet days in a year (i.e., after excluding precipitation less than 0.1 mm). We further break down and more deep analyses are conducted by classifying the stations’ elevation (1000 m ≥ stations’ elevation > 1000 m over the whole country).

The processing stages for error analysis of this study is shown in Figure 2.

4. Results

In the first step we evaluated the daily time-scale of IMERG-V05, -V05-RT, -V06A, MSWEP, ERA5, and SM2RAIN over all days from June 2014 to December 2015 against the in situ observations as reference. Considering that the IMERG-FR V06A started in June 2014 while the eHYD was discontinued by December 2015, the study period was confined to the 17 months between June 2014 and December 2015. It is worth to mention that further work is needed to evaluate the seasonal and inter-annual comparison of these products relying on larger sample data. Figure 3 shows the spatial distribution and average statistical indices (bias, CC, RMSE, and MAE) at daily precipitation time-scale for all products over Austria.

Precipitation types vary across the area. This region is typified by stratiform and convective precipitation, while the west and middle of the area (along with alpine mountains), in addition, is dominated by complex precipitation system due to the orography of the area. In Figure 3 and Figure 4 the spatial distribution and the statistical summary of the metrics for the aforementioned products at a daily and monthly resolution over Austria are shown. According to Figure 3, precipitation shows a weaker correlation to ERA5 and SM2RAIN with mean CC of 0.53 and 0.57, respectively, in compare to MSWEP (CC = 0.86) at daily time-scale. Particularly, over the Alpine mountains both SM2RAIN and IMERG-V05-RT indicated low CC skills, while they showed a better CC over the east and middle of the country.

The CC metric is used to describe the agreement between gridded precipitation products and in situ observations. As can be seen in Figure 3, with respect to CC, MSWEP, significantly yields better than other products in the whole domain in the range of 0.8 to 1 in most pixels. However, ERA5 indicated very low CC over the south and northern part of the country and rather high CC in the area with low altitude. The general performances of the CC for all three versions of IMERG and SM2RAIN are relatively similar and relatively low over the western part, in comparison to MSWEP that several factors could contribute to this lower CC over such areas: a) the topography and climate of the west domain is partly complex, might rise a big challenge for SPE accuracy [12]; b) the GPCC stations that are used for the calibration of IMERG are in monthly time-scale, while in this study the examination of the products are on a daily time-scale, leads the quality of IMERG products being potentially degraded. In general, the MAE and RMSE are significantly higher in the high altitudes and low in low altitudes. This is due to more sensitivity of RMSE, and there were high number of local and heavy convective precipitation events over the high altitudes of Austria.

The general analysis of the results shows that IMERG-V05B, -RT, -V06A, and SM2RAIN and ERA5 have similar scores with respect to MAE and RMSE, although the ERA5 surpass the other products according to the bias skill scores. However, MSWEP indicates a better result according to the error indices with bias, RMSE and MAE of –15 mm, 2.86 mm and 1.08 mm, respectively. This means that in Austria, MSWEP daily precipitation is very close to the in situ precipitation observations among the other recent and state-of-the-art precipitation products. These results are consistent with Beck et al. [22] which determine to underscore the importance of applying daily gauge corrections and accounting for gauge reporting times in compare to monthly gauge corrections. Meanwhile, IMERG and ERA5 indicate relatively higher variety of spatial bias and CC. However, one of the causes of the error of the gridded precipitation products might be their precipitation estimation for a whole pixel once there is a localized precipitation event, particularly in the west part of the area which characterized by complex systems, in some of the stations within. Thereby wrongly assigning the event to unaffected stations [23]. The tendency of reanalysis data to overestimate precipitation frequency might be the cause of ERA5 precipitation overestimation [18,24]. Therefore, after numerous occurrences, this process causes an average areal overestimation/underestimation.

Although all the products exhibited almost similar mean statistical skill scores in overall, regionally there were considerable differences. Compared to MSWEP and IMERG-FR, ERA5, SM2RAIN, and IMERG-V05-RT performed substantially worse over regions of complex terrain [22]. The results suggest that the topography and climate characteristics of the region should be considered when choosing between satellite and reanalysis datasets.

The monthly statistical indices from all precipitation products versus in situ observations are shown in Figure 4 and Figure 5. According to the results of monthly precipitation, although all the examined products indicated a rather close performance to in situ measurements, it is evident that MSWEP, followed by IMERG-V05B and -V06A monthly precipitation compared well to the corresponding in situ measurements. With respect to monthly scale, the CC of MSWEP, IMERG-V05B, -V06A, and SM2RAIN exhibited strong agreement with observations over the whole area. Although IMERG-V05-RT and ERA5 indicated rather good CC for the eastern part of the country, they showed weak performance for the western and southern parts of the region, respectively, which might be due to the effect of relief and complex systems in that area. MSWEP with the skill scores of 6.14 mm, 22.37 mm, 28.29 mm, and 0.93 and ERA5 with −2.08 mm, 43.26 mm, 54 mm, and 0.68 for bias, MAE, RMSE, and CC, respectively determined as the best and worst products. According to bias, the ERA5 strongly overestimated precipitation in the north part of the area and underestimated precipitation in south and west part of the country, with a mean areal bias value of −2.08 mm, that might be due to its native low-resolution and/or parameterization limitation during the precipitation generation processes [17,25]. The box-plots can confirm that most of the IMERG-V05B, -V06A, and MSWEP’s pixels are in a smaller range, close to zero, in comparison to ERA5, with a wider bias range. This suggested that their gauge-correction methodology requires re-evaluation. Overall, MSWEP, followed by IMERG-V05B and -V06A, showed improvements in monthly precipitation in comparison with other products.

Figure 5 shows the daily and monthly time-series comparison and boxplots of regional average precipitation from stations and other precipitation estimate products over Austria from June 2014 to December 2015. All precipitation products generally captured the spatio-temporal precipitation of daily and monthly time-scales, with the highest amount occurred in July and August 2014 and the lowest amount observed in February and December 2015. However, in monthly comparison it is evident that MSWEP estimates are very close to in situ observations and tend to slightly overestimate precipitation during August 2014 and July 2015, which might be due to the small scale of the precipitation systems that are dominant during these months, while IMERG-V05-RT seems to have systematic overestimation from March to November 2014.

According to Figure 5, MSWEP outperformed other products with slight overestimation over only the November 2014 and January 2015. The mean monthly data indicated that SM2RAIN underestimated precipitation during December 2014–April 2015 over Austria, which reflects a possible limitation of SM2RAIN-ASCAT data during the cold months. The SM2RAIN underestimation in winter can be related to snowfall that SM2RAIN is unable to estimate. The behavior of IMERG V05B and -V06A were almost similar with slight overestimation, while a greater overestimation of precipitation is observed mainly in the months with less precipitation intensity. IMERG-V05-RT shows systematic underestimation for February–October 2015. Additionally, from the median and the 25th and 75th percentiles of the box-plots, one can obtain that the precipitation estimated by MSWEP followed by IMERG-V05B and -V06A are more accurate than other products, although MSWEP whiskers extend to the most extreme data points. With respect to the box-plots of daily comparison, SM2RAIN indicated fewer extremes and outliers.

4.1. Elevation

Since Austria characterized by complex terrain and big difference in altitude over the country, annual mean precipitation range significantly varies with elevation and climate conditions. The microclimate can be created due to rapid changes in elevation which cause the obstruct the air mass movement or this rapid changes in elevation can cause the updraft of the air mass over the mountains to create orographic rainfall. Hence, for more deep analysis, the evaluation of the precipitation products was conducted by classifying the stations’ elevation equal or less than 1000 m the stations located in greater than 1000 m altitudes (1000 m ≥ stations’ Elevation > 1000 m) over the whole country in order to account for the effect of topography.

For the first category (1000 m ≥ Elevation), the performance of gridded precipitation products was evaluated by comparing daily data for 642 stations which fell into 502 pixels, while the second category contains 140 stations which fell into 125 pixels. Figure 6 and Figure 7 show the spatial distribution of the statistical indices for all products against in situ observations values for different elevation categories. MAE and RMSE evaluation metrics showed similar spatial patterns, while a sharp contrast from east to west of Austria for both elevation categories is observed, except for MSWEP, which indicated gradual variation. According to CC, MSWEP performed well, followed by IMERG-V05B and -V06A over the whole region, while ERA5, SM2RAIN, and IMERG-V05-RT showed weak CC, respectively, particularly over the alpine valleys.

With the increase of elevation, the mean RMSE, MAE, and bias increase and CC decreases, whereas the bias of IMERG-V05B and -V06A show a decreasing trend with increasing elevation/rainfall. The reason for this difference may be attributed to the cancellation of positive and negative biases, while logically due to high precipitation amounts the error should be higher. In other words, MAE and RMSE measure the absolute error magnitude and bias measure the relative error. The MAE, which evaluates the average magnitude error between precipitation products and in situ observations, were 2.26 mm, 2.2 mm, 2.21 mm, 1.44 mm, 2.59 mm, and 2.57 mm for IMERG-V05B, -V06A, -V05-RT, MSWEP, ERA5, and SM2RAIN, respectively, for the elevation category of less than 1000 m.

Similarly, CC value of the aforementioned products was ≥ 0.5 in the majority of stations with an average value of 0.69, 0.70, 0.64, 0.86, 0.55, and 0.59, respectively, for the stations with less than 1000 m in altitude, while the average CC value of 0.64, 0.66, 0.55, and 0.85 obtained for the IMERG-V05B, -V06A, -V05-RT, and MSWEP, respectively, for the stations located in the high altitudes. The ERA5 and SM2RAIN products failed to capture the observed daily precipitation with CC < 0.5 in most stations over the high altitudes and complex terrains. In general, one can say all products performed better in the low altitudes compared to the high altitudes.

It is notable to mention that inconsistent estimation of the precipitation products (except MSWEP) is possibly due to the rough terrains effect. The overall performance of the precipitation products is lower in the peripheries of the study area where most stations are situated in the mountainous area [26].

4.2. Extreme Events

In this part, R90th index using daily precipitation were examined to characterize the spatial distribution of daily precipitation and its extremes in order to cover the associated uncertainties. The 90% percentile level of wet days (P ≥ 0.1 mm) as the R90th threshold has been used. The resulting threshold for each station and precipitation products are shown in Figure 8. As can be seen, the lengths of extremes are double over the Alpine area in compare to low altitude regions (eastern part of the country). The stations’ R90th showed the maximum values at high-elevation areas and in the west and northwest of the country. This region is considered the Alpine mountains with high mean annual precipitation amounts and has complex precipitation systems. The spatial distribution of the R90th for MSWEP, IMERG-V05B, and –V06a were rather similar with higher number of days for the precipitation threshold above 90th percentile over the south part of the region, which showed the reliability of the estimations of this index. However, the spatial mean value of R90th for MSWEP was more close to the stations. In contrast, ERA5 underestimated extreme events over the big part of the south region, while showed higher number of extreme days over north Austria. Moreover, SM2RAIN, displayed underestimation of the R90th, almost over the whole country, except for some parts over the western region.

Figure 9 illustrates the spatial distributions of aggregated precipitation for stations (through bilinear interpolation), IMERG-V05B, -V06A, -V05-RT, MSWEP, ERA5, and SM2RAIN across Austria during the study period. The high precipitation areas extended from east to west along with alpine mountains. Although there are differences in magnitudes of precipitation among the products, in general, all products reasonably captured the precipitation distribution for most parts of the domain. The remarkable precipitation gradients are well-captured by MSWEP, possibly due to using daily in situ observation for bias correction in its algorithms when compared to IMERG, which uses monthly in situ observations for its bias correction. Another cause might be the native higher spatial resolutions of MSWEP (0.1° × 0.1°) than for example ERA5 product (~0.28° × ~0.28°). Nevertheless, ERA5 only poorly agrees with the gauge-based data at daily and monthly time scales, while patterns of accumulated precipitation agree well.

Moreover, IMERG indicates smoother precipitation trend from the west to the eastern part of the study area. The other reason for less comparable of IMERG products with MSWEP might be due to the limited temporal sampling of observations through active and passive microwave satellite sensors in comparison to MSWEP [27,28]. The station observation shows mean precipitation of 2.78 mm, whereas IMERG-V05B, -V06A, MSWEP, and ERA5 overestimate and IMERG-V05-RT underestimates over the whole domain with the mean precipitation values of 3.08 mm, 3.11 mm, 3.11 mm, 3.36 mm, and 2.41 mm, respectively.

Figure 9 indicates that the total annual precipitation increases with elevation in the center of the country and extended to the west and south parts of the domain. In contrast, it reduces with elevation over the east parts. Stations located in the low altitudes of the eastern and northern parts of the basin receive less precipitation compared to the associated high altitudes.

4.3. Precipitation Detection Capability

The spatial distributions and box-plots of POD and FAR for the light-moderate precipitation range (0.1 mm ≤ P < 10 mm) and heavy precipitation (P > 10 mm) over Austria are shown in Figure 10. As can be seen, all products indicated acceptable skill scores in detecting light-moderate precipitation events. These results underscore the substantial advances in earth system modeling and SPE over the last decade. However, the POD of IMERG-05B has a mean areal value of 0.88, while that of IMERG-V06A has a mean value of 0.90, which shows an improvement of IMERG-V-06A over IMERG-V05B in detecting light-moderate precipitation events. Moreover, ERA5, MSWEP, and SM2RAIN indicated higher average POD with 0.94, 0.93, and 0.93, respectively, than all IMERG products. However, looking at the spatial distribution of POD indicating that the ECMWF’s fourth-generation reanalysis (ERA5) and MSWEP have obvious advantages in detecting light-moderate precipitation events. In general, each value of POD of ERA5 and MSWEP is significantly better than other products, particularly IMERG products, at most of the stations, which might be due to the tendency of reanalysis data to overestimate light-moderate precipitation frequency [18,24]. In contrast, MSWEP is superior to other products to correctly detect precipitation and no-precipitation events.

The FAR for light-moderate precipitation range of all IMERG and MSWEP products presents almost similar spatial distribution pattern with better performance of MSWEP, particularly over the west part of the country. Figure 10 also shows that MSWEP has the lowest average FAR value (0.11) than other products over the area, while SM2RAIN and ERA5 reveal the highest FAR values (0.33 and 0.25).

As can be seen, the FAR values of IMERG products gradually rising from east to west, which indicates that it is more likely to appear false alarm in areas with higher precipitation. In addition, higher FAR might be due to the high amount of moisture in the atmosphere in this area that the satellites observed, although precipitation did not occur because of the evaporation of raindrops before reaching the ground [29].

For the heavy precipitation category (P > 10 mm), MSWEP and SM2RAIN products were found as the most and least powerful products to detect precipitation with the average value of 0.74 and 0.28 for MSWEP and 0.35 and 0.41 for SM2RAIN with respect to POD and FAR values over the area. Compared with MSWEP, IMERG-V05B, and -V06A, the ERA5 product has more complex spatial non-uniformity of POD and FAR. The MSWEP was found to dominate in the east and north, while the ERA5 dominated in the west in detecting the events. Despite this, a low POD and a high FAR of IMERG products for heavy precipitation mean that they are not able to properly detect precipitation in their exact precipitation categories (P ≥ 10 mm), but they might be able to detect the amount of precipitation somewhat lower than the specified intensities [3].

Lower POD for IMERG-V05-RT, particularly over the high altitudes in the west part of the country may be associated with missed precipitation over this region. The missed precipitation may be caused not only by snow cover on the ground at higher altitudes but also precipitation originates from small-scale and short-lived convective systems.

Notice that in the northern region of the study areas the value of FAR for precipitation above 10 mm/day from all IMERG products was rather high, which could be due not only to evaporation and not falling the small and tiny raindrops of the observed liquid water in the atmosphere profiles during the warm seasons, but also short-term precipitation events are highly variable in space and time and might not be detected by rain gauges, while being detected by other gridded products.

5. Discussion

In this research, assessment and comparison of the aforementioned precipitation products have provided insights into how different errors vary with precipitation intensities, elevation, and climate zones. With respect to CC, MSWEP significantly yields better than other products in the whole domain in the range in most pixels. However, ERA5, followed by SM2RAIN, indicated low CC over the southern and western parts of the country and rather high CC in the area with low altitude (Figure 3). The similarities of both IMERG-V05B and -V06A products are very consistent across the scores in daily and monthly time-scales over the whole country. The southern part of the domain is characterized by high positive biases up to ±2 mm/day for IMERG-V05B, -V06A, and MSWEP, whereas bias of the northern and eastern parts are much lower. This can be due to the higher precipitation intensities in these regions, ranging from 1400 mm to 2600 mm during the time period of this study. Moreover, varied orography and complex precipitation processes might be the other reason for this high statistical errors. However, MSWEP significantly performs other products, followed by IMERG-V06 and -V05B products. Moreover, the low bias, RMSE, and MAE and high CC along the eastern part of the domain (Lower Austria) shows an interesting feature. A considerable underestimation of precipitation along the complex train at the southern and western parts of the domain that characterized the complex precipitation is common to IMERG-V05-RT and ERA5 products.

The average monthly data showed that SM2RAIN underestimated precipitation during the cold months over Austria (Figure 5). This underestimation in winter can be related to snowfall and/or frozen soil, which SM2RAIN is unable to estimate. This finding is consistent with those shown by Paredes-Trejo et al. [30], who evaluated the performance of SM2RAIN over Brazil. Moreover, the ERA5 and SM2RAIN products failed to capture the observed daily precipitation with CC < 0.5 in most stations over the high altitudes (elevation > 1000 m) and complex terrains. In general, one can say all products performed better in the low altitudes (elevation < 1000 m) compared to the high altitudes. As the ASCAT soil moisture product has severe limitations over frozen soil, snow-cover, rainforest, and complex topographical regions, SM2RAIN-ASCAT, which is derived based on the ASCAT soil moisture product, also has difficulties to estimate precipitation over these regions [21,31,32,33]. In general, MSWEP properly captured the precipitation gradients, most likely due to using daily in situ observation for bias correction in its algorithms while IMERG uses monthly in situ observation for its calibration (Figure 9). Another cause might be the higher native spatial resolution of MSWEP (0.1° × 0.1°) than for example the ERA5 product (~0.28° × ~0.28°), or higher temporal resolution of MSWEP (3-hourly) in compare to for example SM2RAIN (daily). The algorithm of MSWEP optimally merges the gauge, satellite, and reanalysis precipitation estimates combining the advantages of the different data sources. Moreover, even though at daily and monthly time scales, ERA5 only poorly agrees with the gauge-based data, patterns of accumulated precipitation agree well. The adequate representation of spatial accumulation patterns may be due to (i) the high number of observations assimilated in ERA5, and (ii) assimilate precipitation data from ground-based radar observations (2009 onwards), although ERA5 fails to place the precipitation in the correct areas, when compared with rain gauges.

With respect to detecting light-moderate precipitation events, ERA5, MSWEP, and SM2RAIN indicated higher average POD compared to all IMERG products. For the heavy precipitation threshold (P > 10 mm) MSWEP indicated the most robust and SM2RAIN were found as the less powerful product to detect precipitation with respect to POD and FAR values over the area (Figure 10). The results indicated IMERG-V05-RT (for the entire country, except eastern and southeastern regions), ERA5 the (entire country, except a narrow band in the upper middle of the domain which is extended from east to west), and SM2RAIN (the entire country) are unreliable at detecting precipitation at heavy precipitation category. Regarding the SM2RAIN precipitation products, this can be attributed to soil moisture retrieval errors, which highly affected the estimation of the precipitation quality derived from the SM2RAIN algorithm. SM2RAIN implemented a static correction procedure for climatological correction based on a cumulative density function (CDF) and the ERA5 reanalysis data [34]. Moreover, ERA5 showed some limitation, with emphasis on heavy precipitation, which can additionally affect the quality of SM2RAIN. In, overall, SM2RAIN-ASCAT, ERA5, and IMERG-V05-RT still face a significant challenge to estimate the amount of precipitation, while MSWEP-V2.2 and IMERG-V05B-FR, and V06A-FR revealed good performance to accurately estimate and detect precipitation over Austria. Thus, these products can offer a valuable alternative to in situ measurements for operational use in various applications.

6. Conclusions

To elucidate the strengths and weaknesses of recently released gridded precipitation datasets, we conducted a comprehensive evaluation of the performance of IMERG-FR-V05B, -V06A, IMERG-V05B-RT, ERA5, SM2RAIN-ASCAT, and MSWEP-V2.2 at daily and monthly time-scales for Austria using a dense network of gauges (882 stations) as a reference. The evaluation was carried out based on continuous and categorical statistical metrics for the period June 2014–December 2015. Apart from standard evaluations, we also assessed their performance with respect to the elevation of the stations and extreme events. In agreement with earlier studies, skills vary with respect to elevation, land surface characteristics and snow-rain phase of precipitation. In summary, our study shows that:

• At the daily time-scale, MSWEP shows highest agreement with the gauge-based data followed by IMERG-V06A and -V05A. IMERG-V05-RT, ERA5, and SM2RAIN precipitation products show weaker correlations.
• The skill scores of both IMERG-V05B and -V06A are very similar at daily and monthly time-scales and the newer version of IMERG (-V06A) did not indicate a strong improvement over IMERG-V05B over this area.
• All precipitation products generally capture the spatio-temporal precipitation patterns at daily and monthly time-scales, with the highest precipitation amounts observed in July and August 2014 and the lowest amount observed in February and December 2015.
• At the monthly time-scale, MSWEP precipitation estimates are very close to in situ observations while slightly overestimating precipitation during August 2014 and July 2015. This might be due to the small scale of the precipitation systems that are dominant during this months, while IMERG-V05B, -V06A, and ERA5 seem to have systematic overestimation over the entire months.
• MSWEP outperforms other products. The mean monthly data indicate that SM2RAIN underestimates precipitation during the cold months which might be due to the snowfall that SM2RAIN is unable to estimate. The behavior of IMERG-V05B and -V06A was almost similar with slight overestimation in comparison with in situ observations, while a greater overestimation of precipitation is observed mainly in the months with less precipitation.
• The overestimation or underestimation over the area with complex precipitation systems reveals that the newest generation of the satellite and reanalysis precipitation products examined in this study, even though they are improved, still have difficulties in estimating accurate precipitation amounts in capturing the effects of topography on precipitation.
• For the heavy precipitation category (P > 10 mm), MSWEP and SM2RAIN products were found as the most and least powerful products with the average value of 0.74 and 0.28 for MSWEP and 0.35 and 0.41 for SM2RAIN with respect to POD and FAR values over the area.
• The FAR for light-moderate precipitation (0.1 mm ≤ P < 10 mm) range of all IMERG and MSWEP products presents almost similar spatial distribution patterns with slightly better performance of MSWEP, particularly over the western part of the country.
• Regarding the heavy precipitation (P ≥ 10 mm) detection, MSWEP and IMERG products were found as the most and least powerful products, respectively, with the average value of 0.70 and 0.22 for MSWEP and 0.59 and 0.47 for IMERG-V05B for POD and FAR values over the area. Such robustness may be rooted in applying the daily gauge corrections for MSWEP.
• Inadequate number of gauges, provided by the GPCC and using the monthly gauge correction might be a cause for IMERG products being less powerful compared to MSWEP, which uses daily gauge data.
• The spatial distribution of the R90th for MSWEP, IMERG-V05B, and -V06a were rather similar, which showed the reliability of the estimations of this index. However, the spatial mean value of R90th for MSWEP was closer to the stations. In contrast, ERA5 underestimated extreme events over large part of the south of the country, while showing higher number of extremes over north Austria. Moreover, SM2RAIN underestimates the R90th, almost over the entire country, except for some parts over the western region.
• According to the elevation categories, MAE and RMSE evaluation metrics showed almost similar skills for all products, while a sharp contrast between the east and west of Austria with respect to both elevation categories is observed, except for MSWEP, which indicated gradual variation. With respect to CC, MSWEP performed well, followed by IMERG-V05B and -V06A over the whole region, while ERA5, SM2RAIN, and IMERG-V05-RT showed weak CC, respectively, particularly over the alpine valleys.
• Daily CC values of the all products were greater than 0.5 (CC ≥ 0.5) in the majority of stations with an average value of 0.69, 0.70, 0.64, 0.86, 0.55, and 0.59, for the stations with less than 1000 m in altitude, while the average CC value of 0.64, 0.66, 0.55, and 0.85 obtained for IMERG-V05B, -V06A, -V05-RT, and MSWEP, respectively, for the stations located at high altitudes (stations located above 1000 m asl). The ERA5 and SM2RAIN products failed to capture the observed daily precipitation with CC < 0.5 for most stations at high altitude and in complex terrain.

As expected, all products are able to reproduce the main characteristics of the precipitation in Austria. However, MSWEP performed significantly better than other products, followed by IMERG-V05B and -V06A. Except for MSWEP, the other products indicated difficulty capturing the effects of relief on precipitation over the complex terrain. This research is to our knowledge the first study to evaluate IMERG-V06A, ERA5, MSWEP-V02.2, and SM2RAIN-ASCAT over Austria. Since MSWEP-V2.2 is more robust, statistically, and has long-recorded data (from 1979–2017), we suggest using this product for further studies in climate applications over this region. Moreover, the IMERG data are also available from 2000 to the near present. The inclusion of elevation effects seems to be crucial for a realistic estimation of the spatial distribution of precipitation in mountains areas. We suggest the developers of the examined products highly consider the actual topography, steep terrain, and deep, narrow valleys, over the mountainous area to obtain more realistic precipitation amounts. Moreover, using daily precipitation gauge data, like MSWEP, instead of monthly precipitation data, like IMERG products, can significantly improve the accuracy of the precipitation products. It is worthy to mention that the SM2RAIN-ASCAT product is available only at a daily time step (00:00 UTC), while other products are available at sub-daily time-scales. Therefore, SM2RAIN-ASCAT was evaluated against rain gauges and other products with a few hours’ difference, which may contribute to increased errors in this study. Moreover, in the case of gauge measurements, there might also be some uncertainties and systematic errors due to wind effects, wetting, evaporation, and splashing, which typically amounts to 5%–10% in summer, and even higher in snow conditions [35], which should be considered in further studies. Moreover, we suggest assessing the performance of the IMERG-V06B, which was not yet available during the analysis of this study, and the near-real-time product of MSWEP and use them in hydro-meteorological models. We expect that the regional analysis of the recently released gridded precipitation products revealed in this study can give users a broader perspective and understanding of the features associated with currently precipitation products. It is worth mentioning that the performance ranking of the products may differ across the other regions depending on the amount of gauge data utilized and the quality control applied for each dataset as well.