Multiscale Ground Validation of Satellite and Reanalysis Precipitation Products over Diverse Climatic and Topographic Conditions

Abstract

The validity of two reanalysis (ERA5 and MEERA2) and seven satellite-based (CHIRPS, IMERG, PERSIANN-CCS, PERSIANN-CDR, PERSIANN-PDIR, PERSIANN, and TRMM) precipitation products was assessed in relation to the observations of in situ weather stations installed in different topographical and climatic regions of Pakistan. From 2010 to 2018, all precipitation products were evaluated on daily, monthly, seasonal, and annual bases at a point-to-pixel scale and over the entire spatial domain. The accuracy of the products was evaluated using commonly used evaluation and categorical indices, including Root Mean Square Error (RMSE), Correlation Coefficient (CC), Bias, Relative Bias (rBias), Critical Success Index (CSI), Success Ratio (SR) Probability of Detection (POD), and False Alarm Ratio (FAR). The results show that: (1) Over the entire country, the spatio-temporal distribution of observed precipitation could be represented by IMERG and TRMM products. (2) All products (reanalysis and SPPs) demonstrated good agreement with the reference data at the monthly scale compared to the daily data (CC > 0.7 at monthly scale). (3) All other products were outperformed by IMERG and TRMM in terms of their capacity to detect precipitation events throughout the year, regardless of the season (i.e., winter, spring, summer, and autumn). Furthermore, both products (IMERG and TRMM) consistently depicted the incidence of precipitation events across Pakistan’s various topography and climatic regimes. (4) Generally, CHIRPS and ERA5 products showed moderate performances in the plan areas. PERSIANN, PERSIANN-CCS, PDIR, PERSIANN-CDR, and MEERA2 products were uncertain to detect the occurrence and precipitation over the higher intensities and altitudes. Considering the finding of this assessment, we recommend the use of daily and monthly estimates of the IMERG product for hydro climatic studies in Pakistan.

1. Introduction

Precipitation plays an important role to maintain atmospheric balance. It is an essential parameter of many applications such as climate change studies, crop water requirement, natural hazards analysis, hydrological modeling, and the prediction of floods and droughts [1,2,3]. In general, two sources of precipitation data (weather radars and rain gauges) are considered as the most reliable data sources for various applications [4,5]. However, due to uneven distribution, sparseness of observation data, and the weak relationship between precipitation intensity and topography, it is very difficult to obtain accurate and reliable data in developing regions of the world [6]. Many satellite-based precipitation products (SPPs) have been launched in recent decades to provide worldwide data for several hydrological, environmental, meteorological, and agricultural application [7,8]. The SPPs provide un-interrupted estimates at fine spatial and temporal resolutions [9,10,11]. Other than SPPs, several reanalysis products have also been introduced to overcome the problem of precipitation sparseness [12].

Although SPPs and reanalysis products are capable of providing un-interrupted information of global precipitation at fine spatial and temporal resolutions, their accuracy is dependent on the regional topography and climatic system [13,14,15,16]. Therefore, the assessment of the accuracies of SPPs and reanalysis estimates is very important prior to their direct application for different hydrological and meteorological studies. The validation of SPPs and reanalysis products is also a very important to identify a suitable proxy of gauge-data for crop irrigation scheduling in dry and semi-dry regions with less ground-based gauging stations, such as Pakistan. Most of the recent SPPs provide precipitation estimates using advanced precipitation estimation algorithms, which can provide consistent data at fine spatiotemporal scales using signals from the infrared (IR) and microwave (MW) sensors. The Global Precipitation Mission (GPM) of National Aeronautics and Space Administration (NASA) of the United States of America has introduced the latest SPPs for several hydro-climatic applications. Such products include Integrated Multi-Satellite Retrievals for GPM (IMERG) and Tropical Rainfall Measuring Mission (TRMM). Moreover, Precipitation Estimation from Remotely Sensed Information using Artificial Neural Network (PERSIANN) family’s products are also capable of combining data from IR and MW sensors to provide uninterrupted precipitation estimates. Previously, several researchers evaluated the accuracies of reanalysis and SPPs with reference to the in situ ground-based observations of precipitation in different topographic and climatic regions of the world [8,15,17,18,19,20]. It is well reported that the local topography and climatology greatly influence the performances of SPPs and reanalysis products [6,10,16,21,22]. However, the performances of the latest SPPs and reanalysis products were not characterized and compared over different topographic and climatic regimes of Pakistan. Therefore, this study aimed to investigate and compare the performances of the nine latest precipitation products (CHIRPS, IMERG, ERA5, MEERA2, PERSIANN-CCS, PERSIANN-CDR, PERSIANN-PDIR, PERSIANN, and TRMM) over the diverse climatic and topographic conditions of Pakistan.

Among the considered SPPs, the SM2Rain-ASCAT products are available since 2007. It is a new globally available precipitation product that was developed using the SM2RAIN algorithm to extract precipitation estimates from the ASCAT satellite soil moisture data. The “Climate Hazards Group Infrared Precipitation with Station Data (CHIRPS)” is a quasi-global precipitation dataset that spans 35 years, available since 1981. CHIRPS incorporates in-house climatology satellite imagery (0.05° resolution) and in situ station data to create gridded precipitation time series for trend analysis and seasonal drought monitoring [13,14,23]. PERSIANN-CDR [24] provides daily worldwide precipitation data with a resolution of 0.25°. PERSIANN-CCS is a cloud classification system with inceptions that can be customized. As compared to the traditional constant threshold approach, the variable threshold approach in PERSIANN-CCS enables the detection and separation of cloud computing spots. The IMERG product provides un-interrupted estimates of precipitation using data from the GPM satellite constellation [25,26]. It has a 0.25° × 0.25° resolution and provides quasi-global coverage. ERA5 provides hourly estimates of climate variables at the global scale [27,28]. The Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) provides precipitation data since 1980. MERRA-2 incorporates new features in the GEOS and GSI assimilation models.

Previously, the accuracies of few SPPs, for instance IMERG, TRMM, and PERSIANN-CDR were evaluated in different parts of Pakistan. However, most studies reported the performances of SPPs and reanalysis products over the mountainous parts of the country. In the Hindu-Kush Mountains of Pakistan, for instance, Hamza et al. (2020) investigated the accuracy of two precipitation products (IMERG and TRMM) and suggested using the monthly IMERG product for hydro-climatic applications [29]. Nadeem et al. (2022) evaluated the performances of the PERSIANN product over the Himalayan range of Pakistan [30]. They found significant underestimation of the amount of precipitation represented by the PERSIANN product. Anjum et al. (2018) compared the performances of IMERG with the TRMM products over the northern highlands of Pakistan and highlighted the significant influence of local topography on the performance of both products [31]. Assessments and comparisons of the performances of different SPPs and reanalysis products over the different topographic and climatic regions of Pakistan are lacking until no [29,30,32]. Therefore, the main aim of this study was to evaluate and compare the performances of nine satellites and reanalysis products (PERSIANN-CCS, PERSIANN-CDR, CHIRPS, ERA5, IMERG, MEERA2, PERSIANN-PDIR, PERSIANN, and TRMM) over a wide range of climates and topographies in Pakistan. Due to the diversity of the climate system and complexity of topography, the observations of precipitation from ground-based gauging stations are sparse in Pakistan [26,27,28,33]. It is expected that SPPs or reanalysis products will overcome the problem of sparsity of precipitation data. This will be the first assessment and comparisons of nine precipitation products (SPPs and reanalysis) over diverse topographic and climatic regions of Pakistan.

The assessment methodology is explained in Section 2 of this paper. The study region is described in Section 2.1, and the datasets are discussed in Section 2.2. In Section 2.3, the evaluation and categorical indices that were used to assess the SPPs are described. Section 3 of this paper contains the results of this study. Section 4 includes the discussions of the most important results. The main conclusions from this validation analysis are summarized in the Conclusions (Section 5). The hydrologists, meteorologists, and water managers in Pakistan, as well as the algorithm developers working on global SPPs and reanalysis products, will all find this study’s findings to be useful.

2. Materials and Methods

2.1. Study Region

Geographically, the Islamic Republic of Pakistan is located in South Asia (between 24°N–37°N and 62°E–75°E). Its total area is approximately 881,913 km² that is mainly divided in three distinct regions: Baluchistan Plateau; Indus River (covers provinces of Punjab and Sindh); and Northern highlands (Khyber Pakhtunkhwa (KPK), Azad Jammu Kashmir (AJK), Gilgit Baltistan, and the capital of Pakistan “Islamabad”). The annual precipitation varies greatly over the entire country. The northern highlands of country receive very heavy precipitation during the winter season (December–February), whereas the plains of Punjab received 50–70% of precipitation during the summer season (June–September). The gradient of altitude varies from north to south, with higher altitudes in the northern parts and lower altitudes in the southern parts. The topographical map of Pakistan is shown in Figure 1.

2.2. Datasets

In Pakistan, gauge-based rainfall datasets are very limited and maintained by various Pakistan Meteorological Department weather stations. Daily meteorological data from 110 Pakistan Meteorological Department (PMD) weather stations were collected. However, for this assessment, the daily observations of only 61 meteorological stations were found to be reliable due to the missing values (more than 20%) in other datasets. Depending upon the reliability and quality of the daily datasets of the in situ stations considered herein, the validation of the considered precipitation products was restricted for the period of 2010–2018.

Table 1. Salient features of the gauging stations considered for the validation of products.

Serial Number	Station	Lat	Long	Altitude (m)	Average Precipitation (mm)
1	Astore	35.37	74.90	2168.0	420
2	Balakot	34.38	73.35	981.0	1302
3	Bunji	35.67	74.63	1470.0	178
4	Burzil	34.91	75.09	4030.0	749
5	Chillas	35.42	74.10	1251.0	277
6	Chitral	35.85	71.83	1500.0	432
7	Dir	35.20	71.85	1370.0	1303
8	Drosh	35.57	71.78	1465.0	510
9	G-Dopata	34.20	73.60	813.5	1359
10	Gilgit	35.92	74.33	1457.2	168
11	Gupis	36.17	73.40	2156.0	174
12	Jhelum	32.93	73.73	287.2	834
13	Kakul	34.18	73.25	1309.0	1288
14	Khot	36.52	72.58	3505.0	610
15	Kotli	33.52	73.89	614.0	1233
16	Mangla	33.13	73.63	305.0	943
17	Murree	33.92	73.38	2127.0	1627
18	Muzaffarabad	34.40	73.50	702.0	1384
19	Peshawar	34.00	71.93	327	488
20	Ratu	35.15	74.81	2920.0	662
21	RawlaKot	33.87	74.27	1677.0	1219
22	Skardu	35.34	75.54	2316.5	243
23	S-Sharif	34.82	72.35	970.0	986
24	PBO. Nawabshah	26.25	68.36	37	219
25	PBO. Panjgur	26.96	64.1	968	76
26	PBO. Pasni	25.26	63.48	9	223
27	M.O. Badin	24.63	68.9	9	255
28	Padidan	26.85	68.13	46	122
29	Rohri	27.66	68.9	66	110
30	Hydrabad	25.38	61.8	28	90
31	JACOBABAD	28.3	68.46	55	192
32	Karachi Airport	24.9	66.93	22	138
33	NawabShAh	26.25	68.36	37	170
34	Larkana	27.53	68.23	52.7	116
35	Rohri2	27.66	68.9	66	84
36	Bahawal Nagar	30	73.24	307	321
37	Bahawal Pur	29.33	71.783	110	188
38	Bahawal Pur(A/P)	29.383	71.683	119	213
39	Bhakkar	31.616	71.06	162	366
40	Noorpur Thal	31.866	71.9	186	558
41	Jauharabad	32.5	72.43	187	461
42	Faisalabad	31.43	73.13	186	446
43	Jhelum2	32.93	73.73	287	855
44	Khanpur	28.65	70.683	88	254
45	Lahore A.P.	31.583	74.4	216	812
46	Multan	30.2	71.43	122	257
47	Mandi Bahauddin	32.96	73.8	253	779
48	Sialkot	32.516	74.53	255	1025
49	Sialkot Airport	32.53	74.03	240	933
50	Sargodha	32.05	72.66	187	545
51	Toba Tek Singh	30.983	72.783	155	363
52	D.G. Khan	30.05	70.63	148	251
53	Jhang	31.26	72.316	158	405
54	Mangla2	33.06	73.63	283	943
55	Sahiwal	30.65	73.16	172	350
56	Chakwal	32.916	72.85	519	669
57	Gujranwala	32.36	74.35	227	858
58	Okara	30.8	73.43	180	421
59	Rahim Yar Khan	28.43	70.316	83	157
60	Gujrat	32.56	74.06	240	793
61	Rawalpindi	33.56	73.02	1271	1308

presents the details of selected meteorological stations considered for this validation study.

All precipitation products were downloaded from their official websites for the period of January 2010– December 2018. The estimates of IMERG at 0.1° × 0.1° spatial resolution were acquired from (“http://pmm.nasa.gov/data-access/downloads/gms/ (accessed on 15 July 2021)”). The estimates of the TRMM product at 0.25° × 0.25° scale were downloaded from (“http://disc2.nascom.nasa.gov/tovas/ (accessed on 20 August 2021)”). Most of the recent SPPs provide precipitation estimates using advanced precipitation estimation algorithms, which can provide consistent data at fine spatiotemporal scales using signals from the infrared (IR) and microwave (MW) sensors. The Global Precipitation Mission (GPM) of National Aeronautics and Space Administration (NASA) of the United States of America has introduced the latest SPPs for several hydro-climatic applications. Such products include Integrated Multi-satellite Retrievals for GPM (IMERG) and Tropical Rainfall Measuring Mission (TRMM) daily estimates of PERSIANN’s family products (“https://chrsdata.eng.uci.edu/ (accessed on 21 August 2021)”). Two CHRS data products (PERSIANN and PERSIANN-CDR) have a spatial resolution of 0.25° × 0.25° and cover the latitude range of 60°S to 60°N. For analyzing daily precipitation variations and trends, the PERSIANN-CDR product provides a dependable, high-resolution, and long-term global precipitation database. The PERSIANN- CSS system covers latitudes from 60°S to 60°N and has a spatial resolution of 0.04° × 0.04°. The PERSIANN-CCS algorithm can be used to categorize cloud-patch features. PDIR-Now has a very low latency from the time of the precipitation event (15–60 min) due to the sampling of IR imagery. CHIRPS-2.0 products estimates were downloaded from its official website (“https://data.chc.ussb.edu/products/CHIRPS-2.0/ (accessed on 17 September 2021)”). ERA5 provides hourly estimates of climate variables at global scale (30 km). The ERA5 data were downloaded from its official website (“https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5/ (accessed on 24 September 2020)”).The Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) provides precipitation data since 1980 at 0.5° spatial resolution downloaded from its official website (“https://gmao.gsfc.nasa.gov/reanalysis/MERRA-2/. MERRA-2/ (accessed on 2 October 2021)”) incorporates new features in the GEOS and GSI assimilation models. Monthly, seasonal, and yearly data series were derived from daily estimates.

2.3. Methods

The ground validation of the estimates of nine precipitation products (IMERG, TRMM, PERSIANN-PDIR, PERSIANN, PERSIANN-CCS, PERSIANN-CDR, CHIRPS, ERA5, and MEERA2) was performed using the observations of gauging stations. In this validation, the methodology of previously published studies [30,34] was used. Only those grids of SPPs and reanalysis products were considered that contained at least one reference gauging station. The uncertainties in the estimates of considered reanalysis and SPPs were evaluated and compared with each other at multiple spatial and temporal scales. Errors in the products at assessed at grid to point level and the entire spatial domain on daily to annual temporal scales. The spatial distribution map of observed precipitation (as shown in Figure 2) was developed using the Kriging spatial interpolation technique. This geospatial technique is recommend for those areas where reference data are nonuniform [32].

Initially, the skill of each product to characterize the spatio-temporal distribution of observed precipitation over the entire country was evaluated. Then, the validation of all considered products at point and entire spatial scales was performed with reference to the observations of 61 gauging stations. For this purpose, previously recommended evaluation indices, including “Bias, relative Bias (rBias), Root Mean Square Error (RMSE), and Pearson Correlation Coefficient (CC)”, and categorical indices, including “Probability of Detection (POD), False Alarm Ratio (FAR), Critical Success Index (CSI), and Success Ratio (SR)” were used [26]. The evaluation indices were employed to validate the accuracy of precipitation products (SPPs and reanalysis products) at daily-to-annual scales. The categories indicators were used to evaluate the product’s ability to identify and represent precipitation.

The dimensionless CC was used to evaluate the linear agreements between the reference observations and the estimates of precipitation products. The bias was calculated to check the discrepancies between the observed (reference) and estimated (satellite and reanalysis) data. The rBias (%) was estimated to assess the relative disagreement (over/under-estimation) of observed precipitation by the SPPs and reanalysis products. The average amount of error was estimated by calculating the RMSE (mm/time) of precipitation products with reference to the gauge-based data. The equations of CC, Bias, rBias, and RMSE are given below:

$$\mathrm{CC}=\frac{{{\displaystyle \sum }}_{i=1}^n\left( Gi-G\right)\left( Ei-E\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(Gi-G\right)}^2 } \times \sqrt{{{\displaystyle \sum }}_{i=1}^n{\left( Ei-E\right)}^2}}$$

(1)

$$\mathrm{BIAS}=\frac{{{\displaystyle \sum }}_{i=1}^n\left(Ei-Gi\right)}{n}$$

(2)

$$\mathrm{r}-\mathrm{Bias}=\frac{{{\displaystyle \sum }}_{i=1}^n\left(Ei-Gi\right)}{{{\displaystyle \sum }}_{i=1}^{n}Gi}\times 100$$

(3)

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n {\left(Ei-Gi\right)}^2}$$

(4)

where Gi represents the data of reference gauges, G shows the average of the gauge data, Ei denotes the estimates of satellite/reanalysis product, and E indicates the mean of the estimates of satellite/reanalysis product. The term “n” refers to the total number of datasets. Precipitation products can be used as a substitute for gauge-based data when their CC value is near to 1, and their Bias and RMSE values are near to zero. The acceptable range of the r-Bias and CC for the use of SPPs/reanalysis data in hydrological studies is ±10.0% and 0.7, respectively. The categories indices were computed using the equations listed below:

$$\mathrm{POD}=\frac{H}{H+M}$$

(5)

$$\mathrm{FAR}=\frac{F}{H+F}$$

(6)

$$\mathrm{CSI}=\frac{H}{H+M+F}$$

(7)

where H represents the number of precipitation events that the SPPs/reanalysis products accurately reported, F represents the number of precipitation events that the SPPs/reanalysis products misrepresented, and M represents the number of precipitation events that the reference gauging stations observed but that the SPPs/reanalysis products missed. The perfect value of CSI and POD is one, whereas the perfect value of the FAR is zero. Figure 3 illustrates the layout of the adopted steps to assess the performances of SPPs and reanalysis precipitation products over Pakistan.

For a more precise evaluation of the SPPs and reanalysis products, the probability density function (PDF) of the daily estimates of in situ gauging stations and nine precipitation products (3PERSIANN-CCS, PERSIANN-CDR, CHIRPS, ERA5, IMERG, MEERA2, PERSIANN-PDIR, PERSIANN, and TRMM) was examined at various thresholds. The PDFs were analyzed by following a previous study [17]. The thresholds for daily precipitation rates were established in accordance with the recommendations of the “World Meteorological Organization (WMO)”. Additionally, the Taylor Diagrams (Taylor, 2001) were developed to elaborate the summary of the performances of the considered precipitation products. The daily and monthly estimates of nine different precipitation products were used to create these graphs, and they were created with reference to the gauge-based data from Pakistan. Several previous assessments [27,28] of precipitation products had used the Taylor Diagram to summarize the accuracies of different precipitation products.

3. Results

3.1. Potentials of Precipitation Products to Represent the Spatio-Temporal Distribution of Precipitation

Comparison of the spatial variability in the amount of average daily precipitation obtained from the nine precipitation products and the reference stations is shown in Figure 4. In Pakistan, the northern high lands generally received a higher amount of precipitation. It was found that the major portion of the annual precipitation in the northern parts of the country was received during the winter season, while 50–70% of the total precipitation in the Punjab Province was occurred during the summer season. Considering the gauge data, higher variability in the amount of average daily precipitation was found in the northern parts as compared to the southern parts of the country, where daily rainfall is very low. The results reveal that the IMERG and TRMM products outperformed other SPPs and reanalysis products in terms of their capability to represent the estimated amount of daily precipitation variability throughout the entire country. Under humid climatic conditions (northern parts of the country), the PERSIANN-CCS, PERSIANN-CDR, PERSIANN-PDIR, PERSIANN, MEERA2, and ERA5 products showed a poor performance in terms of ability to accurately represent the variability of daily precipitation amount.

In Figure 5, the temporal variability of the average amount of daily precipitation obtained from the in situ gauges is compared to that of all satellite-based and reanalysis data for the years 2010–2018. The time-series estimates of the average daily precipitation amount were derived using the moving average of daily data of all sources. Similar methods were employed by researchers to depict uncertainty in the estimation of daily precipitation amounts by SPPs in China’s Tianshan Mountains and the northwestern part of South America [10,35,36,37]. Generally, the reference data indicate two peaks in the precipitation time series data, as illustrated in Figure. PERSIANN-CCS, PERSIANN-CDR, PERSIANN-PDIR, PERSIANN, and reanalysis products indicate poor performances in terms of skill to track the temporal variability of observed precipitation over the study domain. However, the IMERG, TRMM, and CHIRPS products performed better in terms of their capacity to follow the temporal variability of average daily precipitation. The temporal variability of precipitation was not tracked by ERA5, whereas MEERA-2 showed an overestimation of the precipitation amount during both high precipitation periods (February–May and August–November).

3.2. Performances of SPPs and Reanalysis Products on Monthly Scale

The comparison of the performances of all reanalysis and SPPs (CCS, CDR, CHIRPS, ERA5, IMERG, MEERA2, PDIR, PERSIANN, and TRMM) products at monthly scales is presented in the Taylor Diagram (Figure 6). To create the Taylor diagram, data of all sources (reference stations, SPPs, and reanalysis products) were normalized. The monthly data of seven products (IMERG, TRMM, CDR, CHIRPS, ERA5, MEERA2, and PDIR) showed better agreements with the reference monthly data, as indicated by higher values of CC (>0.70). However, two products (CCS and PERSIANN) showed poor linear agreements with the reference data (CC < 0.70). In the Taylor Diagram, straight blue lines denote the values of CC. The standard deviation (SD) of the monthly estimates of all data sources (gauges, SPPs, and reanalysis products) was comparable, and the SD values are indicated by circular dotted lines (with black color) in Figure 5.

Figure 7 depicts the fluctuations in the estimated CC and rBias values for nine precipitation products. Overall, both products of GPM (IMERG and TRMM) showed better performances as compared to all other considered products, as witnessed by the box plots of CC for both products. In case of rBias, the ERA-5 reanalysis products showed a relatively poor performance as compared to the other products, indicated by the box plot of rBias of ERA-5 product. Among all considered products, ERA-5 showed the maximum underestimation of the precipitation amount.

3.3. Assessments of Precipitation Products on Daily Scale

Figure 8 displays the summary of the performances of all precipitation products (CCS, CDR, CHIRPS, ERA5, IMERG, MEERA2, PDIR, PERSIANN, and TRMM) against gauged-based daily data in Pakistan. The values of the estimates of CC for IMERG, TRMM, CDR, CHIRPS, ERA5, MEERA2, PDIR, PERSIANN, and CCS are 0.79, 0.70, 0.47, 0.39, 0.62, 0.63, 0.36, 0.26, and 0.11, respectively. This revealed poor agreement between daily gauge-bases data and all SPPs except IMERG and TRMM. In terms of CC, only TRMM and IMERG are in better agreement with gauge data. Similarly, in Figure 8, the box length of IMERG and TRMM showed good agreement >0.7 with the reference daily data. Box of CC for the CCS product indicates its poor performance of daily scale. In terms of rBias, the CCS products indicated maximum inconsistency in the estimated values of rBias, as witnessed by the maximum length of its box as compared with the boxes of other products. The box plot of the ERA-5 product showed the maximum underestimation of the daily precipitation amount by this product. Overall, the IMERG and TRMM products showed better performance on a daily scale.

Figure 9 depicts the influence of precipitation intensity on the evaluation indices estimated for all precipitation products (CCS, CDR, CHIRPS, ERA5, IMERG, MEERA2, PDIR, PERSIANN, and TRMM). Errors in the estimates of all products increased as the precipitation intensity increased, as demonstrated by greater values of RMSE at high rates of precipitation. Conversely, a direct relationship between the values of CC and precipitation intensity was found, which revealed a better linear relationship between the reference data and estimated data from the reanalysis products and SPPs at higher precipitation rates. The increased rate of precipitation led to a reduction in the level of uncertainty in the estimated total amount of precipitation that was acquired from all of the products.

The influence of elevation on the performance evaluation indices of precipitation products (CCS, CDR, CHIRPS, ERA5, IMERG, MEERA2, PDIR, PERSIANN, and TRMM) with reference to the gauging stations is shown in Figure 10. Generally, the CC values of all considered products were inversely proportional to the elevation. This highlights the poor performances of precipitation products at the higher altitudes. The values of Bias were generally not influenced by the change of elevation, as shown in Figure 11. It was also found that the error in the estimates of all products was increased with the increase in elevation. That might be due to the tough topography at higher altitudes.

Figure 12, Figure 13 and Figure 14 show the spatial variability of evaluation indices (CC, Bias, and RMSE) computed for CCS, CDR, CHIRPS, ERA5, IMERG, MEERA2, PDIR, PERSIANN, and TRMM. We identified a wide range of changes in the evaluation indicators across the entire study area. The spatial variation of CC for all PERSIANN’s family products indicated significantly less variations in their CC values over the entire spatial domain. The CC values varied more widely in the IMERG and TRMM than in other products. The regions of the country with the highest precipitation rates typically have the higher values of CC. In terms of a linear relationship with the gauge-based data, ERA5 and MEERA2 products performed better than the CHIRPS and PERSIANN products, as demonstrated by the higher CC values of both reanalysis products.

Figure 15 illustrates the seasonal variability of rBias (%) determined for all SPPs and reanalysis products across the whole study domain. In the winter season, two products—CCS and PDIR—showed the best performance, as indicated by the lowest values of rBias for both products. Conversely, CCS showed the worst performance in the spring season, with the highest overestimation value (rBias > 60%). In the summer season, all precipitation products revealed an underestimation of the precipitation amount. In this season, only two products (CHIRPS and MERRA-2) showed relatively acceptable values of rBias (<10%). ERA-5 showed a significant underestimation of the observed precipitation amount in all seasons, with the worst performance in the summer season (underestimation > 70%). Overall, CHIRPS product showed best performance in terms of relative bias, followed by IMERG.

3.4. Ability of SPPs and Reanalysis Products to Detect the of Precipitation Events

Figure 16 demonstrates the summary of the skills of all considered precipitation products in terms of the probability of detection (POD) in Pakistan. Using the categories indices, Roebber [30] developed a performance diagram to visually compare the findings of the categorical indices. This shows how reference gauges and SPPs/reanalysis products are related spatially. Some previous studies have used this performance diagram to show how different precipitation products performed over diverse topographical and climatic conditions. The estimated value of POD for the IMERG, TRMM, CDR, CHIRPS, ERA5, MEERA2, PDIR, PERSIANN, and CCS was 0.73, 0.62, 0.56, 0.57, 0.44, 0.51, 0.52, 0.36, and 0.44, respectively. The POD was maximum for IMERG, which indicated that the ability of the product to detect the occurrence of precipitation was good, and that the SPP was capable of detecting the occurrence of the majority of spells. As compared with the other SPPs and reanalysis products, the probability of the detection of PERSIANN and CCS products was lowest, however, their success ratio to represent the observed precipitation was good.

The skills of the considered precipitation data products to correctly identify the precipitation events in different seasons was elaborated in performance diagrams (Figure 17). In all seasons, the overall performance of IMERG was superior to that of the other products in terms of its capacity to identify the occurrence of precipitation (spring (POD = 0.78), winter (POD = 0.78), summer (POD = 0.78), and autumn (POD = 0.69)). The POD of TRMM was also good in autumn and winter. PDIR showed unsatisfactory performance in all seasons. The performance of MERRA2, in terms of seasonal POD, was better than CDR, CHIRPS, ERA5, PDIR, and PERSIANN.

The probability density function (PDF) of the observed daily precipitation records (obtained from the gauging stations) nine precipitation products (IMERG, TRMM, CCS, CDR, CHIRPS, ERA5, MEERA2, PDIR, and PERSIANN) is shown in Figure 18. The light precipitation events (<2 mm/day) events were most frequent (approximately 72% of all events) over the entire study duration, as indicated by the datasets of all sources. MERRA2 products showed a significant underestimation of precipitation at all thresholds. ERA5 indicated the overestimation of light precipitation events but underestimated the moderate and heavy precipitation events. CCS showed a considerable overestimation of moderate events. Generally, IMERG and TRMM products revealed a better performance in tracking the precipitation events at different thresholds.

In winter, the PDIR product showed a significant overestimation of slight precipitation events. Conversely, MEERA2 indicated a significant underestimation of light events. Although the performance of the CCS product to represent the occurrence of light events was good, it showed an overestimation of moderate-to-heavy precipitation events. The ERA5 product outperformed all other products this season in terms of its ability to detect the occurrence of precipitation events in Pakistan. In the spring season, the performance of IMERG was better than that of all other products. Generally, all other products were uncertain in their representation of the light-to-moderate precipitation events in the spring season. In the summer season, ERA5 totally failed to present the occurrences of light-to-heavy precipitation events. The IMERG product was well trained to track the occurrences of light-to-heavy precipitation events in the autumn season.

4. Discussion

In this study, several evaluation and categorical indices were used to assess the performances of nine reanalysis and satellite-based precipitation products (ERA5, MEERA2, IMERG, TRMM, CCS, CDR, CHIRPS, PDIR, and PERSIANN) in Pakistan. This analysis was based on the daily, monthly, seasonal, and annual data collected from 61 in situ gauging stations from 2010 and 2018. The analysis of satellite and reanalysis data products showed that their performance was strongly influenced by the climatic and topographic conditions. Several researchers have previously assessed the accuracy of various SPPs (such as PERSIANN, PERSIANN-CCS, and IMERG) [17,18,24,27,38,39] in diverse climatic and topographic regions around the world. The results of previous studies showed that the performance of satellite based and reanalysis precipitation products are dependent on the in situ climatic and topographic conditions [4]. For instance, Yang et al. (2019) assessed the uncertainly in the estimates of satellite-based precipitation products over different climatic and topographic regions in Punjab province of Pakistan [37]. They reported that the accuracy of SPPs significantly varied with the change in climatic and topographic regimes. Hamza et al. (2020) assessed and compared the accuracies of IMERG and PERSIANN-CDR products on diverse topographic regimes in Hindukush South Asia and reported that the performance of both products was greatly influenced by the local topographic conditions [29]. We also found considerable variability in the accuracies of SPPs and reanalysis over different topographic and climatic regions of Pakistan, which is consistent with the findings of [37,40].

As compared with the other SPPs, the estimates of IMERG and TRMM were more accurate in terms of replicating the reference data, which is consistent with the previous findings of [8,31,38]. The IMERG and TRMM products were also capable of representing the spatio-temporal distributions of precipitation over Pakistan. This might be due to the better morphing techniques in both products. This study’s findings are in line with earlier ones, which found that monthly product predictions are more consistently reliable than daily estimates [17]. In terms of CC, PERSIANN-CCS, PERSIANN, CDR, and PDIR showed a poor performance. The seasonal performance of IMERG and TRMM was considerably better than that of other precipitation products.

5. Conclusions

The assessment of nine satellite-based and reanalysis precipitation products (CCS, CDR, CHIRPS, ERA5, IMERG, MEERA2, PDIR, PERSIANN, and TRMM) was done on multiple temporal (daily to annual) and spatial (station and regional) scales, using the data of 61 in situ gauges in Pakistan. All SPPs were evaluated for the period of 2010–2018. The important findings of this study are as follows:

• The spatial variability of precipitation in Pakistan was well-depicted by the IMERG and TRMM products. MEERA2, as well as the PERSIANN family of products, were unsuccessful in monitoring the spatial patterns of precipitation.
• The daily variability in precipitation could be tracked using both the IMERG and TRMM products. However, PERSIANN, PERSIANN-CCS, PDIR, PERSIANN-CDR, or MEERA2 could not adequately describe the temporal variability of precipitation.
• As compared to daily scale, the overall performance of all SPPs and reanalysis products was significantly improved when evaluated on a monthly scale.
• ERA-5 showed a significant underestimation of the observed precipitation amount in all seasons, and showed worst performance in the summer season (underestimation >70%). Overall, during the summer season, the CHIRPS product showed the best performance in terms of relative bias.
• The POD was maximum for IMERG (0.73), which indicated that the ability of the product to detect the daily occurrence of precipitation was very good as compared to other SPPs and reanalysis products.
• In all seasons, the overall performance of IMERG was superior to that of the other products in terms of its capacity to identify the occurrence of precipitation (spring (POD = 0.78), winter (POD = 0.78), summer (POD = 0.78), and autumn (POD = 0.69)).
• The performance of the ERA5 product was comparatively good over the plane topography as compared to rugged topographic conditions.
• The light precipitation events (<2 mm/day) events were at their most frequent (approximately 72% of all events) over the entire study duration, as indicated by the datasets of all sources. Generally, the IMERG and TRMM products revealed a better performance in tracking the precipitation events at different thresholds.
• Only the PERSIANN-CCS product showed significant overestimation ((17.47%) of observed daily precipitation amount—whereas PERSIANN, PDIR, TRMM, ERA5, and MEERA2 showed a significant underestimation of the daily precipitation amount (−11.5%, −12.5%, −15.5%, −40.5%, and −22.15%, respectively). The Bias of IMERG product on daily and monthly scales was with an acceptable range (±10%).

The findings of this validation study advocate the better performance of an IMERG product than all other SPPs in Pakistan under various topographical and climatic conditions. The IMERG outperformed all other SPPs and reanalysis products (PERSIANN’s family, CHIRPS, ERA5, MEERA2, and TRMM for the whole country Pakistan. Moreover, in both daily and monthly scales, the correlation coefficient (CC) for IMERG was (>0.70). Moreover, the estimated relative Bias was also within the adequate limit (±10). Therefore, we advise the use of daily and monthly estimates of the IMERG product for hydro-climatic applications in the Pakistan. Moreover, the data users may also use the monthly estimates of TRMM products in different studies. Findings of this research will be very helpful for hydrologists, meteorologists, and water manages in Pakistan, as well as algorithms developers of SPPs and reanalysis products.