Proxy Data of Surface Water Floods in Rural Areas: Application to the Evaluation of the IRIP Intense Runoff Mapping Method Based on Satellite Remote Sensing and Rainfall Radar

Abstract

Along with fluvial floods (FFs), surface water floods (SWFs) caused by extreme overland flow are one of the main flood hazards occurring after heavy rainfall. Using physics-based distributed hydrological models, surface runoff can be simulated from precipitation inputs to investigate regions prone to soil erosion, mudflows or landslides. Geomatics approaches have also been developed to map susceptibility towards intense surface runoff without explicit hydrological modeling or event-based rainfall forcing. However, in order for these methods to be applicable for prevention purposes, they need to be comprehensively evaluated using proxy data of runoff-related impacts following a given event. Here, the IRIP geomatics mapping model, or “Indicator of Intense Pluvial Runoff”, is faced with rainfall radar measurements and damage maps derived from satellite imagery and supervised classification algorithms. Six watersheds in the Aude and Alpes-Maritimes departments in the South of France are investigated over more than 2000 km² of rural areas during two flash-flood events. The results of this study show that the greater the IRIP susceptibility scores, the more SWFs are detected by the remote sensing-based detection algorithm. Proportions of damaged plots become even larger when considering areas which experienced heavier precipitations. A negative relationship between the mean IRIP accumulation scores and the intensity of rainfall is found among damaged plots, confirming that SWFs preferably occur over potentially riskier areas where rainfall is lower. Land use and soil hydraulic conductivity are identified as the most relevant indicators for IRIP to define production areas responsible for downslope deteriorations. Multivariate logistic regression is also used to determine the relative weights of upstream and local topography, uphill production areas and rainfall intensity for explaining SWF occurrence. This work overall confirms the relevance of IRIP methodology while suggesting improvements to its core framework to implement better prevention strategies against SWF-related hazards.

1. Introduction

With more extreme precipitations expected in the 21st century due to climate change [1,2], increased attention has to be paid to the understanding and modeling of floods as of now. Comparably to river overflowing, inland flood events occurring outside the vicinity of active waterways have had devastating effects worldwide in the past decades. They have been observed all around Europe, both in urban and rural areas, and not only around the Mediterranean border where severe storms happen on a more frequent basis (see [3] for some examples). In France and Switzerland, surface water floods have been estimated accountable for respectively 50% and 45% of all flood damage claims in the past years [3,4]. Triggered by more or less intense rainfall, such floods can appear anywhere on a given territory depending on land use, soil type and topography. In this work, the phrase surface water floods (SWFs [3]) will be referred to when considering the various observed impacts of intense overland flow of rainwater on the land surface, as opposed to fluvial floods (FFs) which are confined to the proximity of streams. Another commonly used term is pluvial flood, which is defined as the result of an extreme precipitation event that creates a flood independent of an overflowing water body. Even though some differences might exist within these two terminologies (pluvial floods are often used in the urban context), both will be used as synonyms here. However, this paper will only focus on rural areas, in relation with the implemented methodology (see Section 2 for details).

Contrary to FFs which can be thoroughly studied in instrumented basins thanks to accurate time series of streamflow, data associated with SWFs are much harder to come by. Scarcity of observational data concerning intense overland flow [5,6,7] assumably stems from multiple combined factors: (i) its potential to appear anywhere in a given area; (ii) its ephemeral nature and (iii) the small spatial scale over which it can materialize [3,8,9]. Indirect measures of SWF impacts are thus used to estimate their occurrence in space and time as well as their severity. We refer to these types of observation as “proxy” data of intense runoff-related impacts. Many proxies exist. Some are obtained from field missions providing ground truth information. Others consist in damage claims contained in insurance records and disaster databases (both agricultural and urban). The latter have been fairly used in the literature to both analyze the spatio-temporal likeliness of SWF occurrence and the validation of pluvial flood prediction models (e.g., [4] in France, but for both FFs and SWFs, [3] in Switzerland and [10] in the Netherlands). Other proxy sources include aerial photos, videos, interviews or local reports. However, none of these sources can be considered completely exhaustive. Insurance records, depending on country legislation and structure, tend to be a very profitable and relevant tool, but suffer from spatial biases towards economically valuable zones [11]. Hence, SWF occurring in uninhabited and non-cultivated areas are not recorded. With the growing availability of satellite imagery at high spatial resolution and high revisit frequency, the detection of SWF footprints on the ground becomes more and more accessible, at least in rural areas [11]. This paves the way for large scale, exhaustive, automatic and repeatable identification of land surface damages in the following days of an extreme weather event, both for operational needs (damage assessment and restoration works) and evaluation of predictive hydrological models.

Among the great variety of hydrological models, some were specifically designed to study and simulate overland flow and intense runoff-related processes such as erosion and sediment yield [12]. Some are physics-based, such as KINEROS [13,14], conceptual, e.g., CEQEROSS [15], or based on a hybrid approach (LISEM [16] and WEPP [17]). The 2D hydraulic FloodAreaHPC model was for instance used for pluvial flood hazard public mapping by Tyrna et al. [18] in Germany. Several geomatics methods have also been developed to map the risks associated with intense runoff. They mostly rely on decision trees depending on topography, land use and soil type, and do not perform any hydrological simulation per se (e.g., STREAM [19]). Other more urban-specific approaches exist. They usually associate susceptibility maps to vulnerability indicators specific to urban networks (population density, buildings distribution, role of various infrastructures). Other flood models were tested in this context, for example on small scale events in Switzerland [20,21]. Bernet et al. [20] assessed the relevance of four models (among which the FLO-2D distributed physics-based model and the MFD geomatics method) to predict SWF occurrence in rural areas while Zischg et al. [21] investigated the performance of the BASEMENT 2D inundation model rather in urbanized areas. Various sources such as event documentations, field observations, aerial photos or insurance claim data were used for performance evaluation.

The IRIP method (French acronym for Indicator of Intense Pluvial Runoff) [12] constitutes a static approach to mapping susceptibility towards the three processes associated with overland flow: production, transfer and accumulation. Without explicit hydrological modeling, IRIP simply consists in building score maps ranging from 0 (no susceptibility) to 5 (high susceptibility) through the attribution of binary indicators (0 or 1 value) representing various spatially distributed landscape factors such as topography, land use and soil type. Contrary to most distributed hydrological models, IRIP does not rely on an extended set of input parameters and does not require any calibration. Since this method does not simulate runoff dynamics per se, its main goal is rather to allow for the implementation of quick and large-scale preventive strategies against intense runoff hazards. In order to be fully operational, evaluation using observational data is crucial. If spatially comprehensive in situ measurements of overland flow are burdensome to come by, Dehotin et al. [22] managed to implement a field network to monitor rainfall events, soil water content and surface runoff occurrence during a whole year at nine locations in a small rural watershed. Along with multi-factor analysis of runoff occurrence, they confirmed that accumulation and production-prone areas in the IRIP susceptibility maps were associated with a higher surface runoff frequency (see Figure 18 of [22]). However, they did not investigate further the relationship between runoff occurrence and IRIP scores. In other works, risk regulatory zonings of surface runoff and soil erosion as well as post-event survey data of impacts on infrastructures and transport networks such as roads and railways were used to evaluate the IRIP model on several case studies in France [23,24,25]. IRIP was also applied in the same context to other regions (near Antibes in [26] and the Bayonne area in [27]). Recently, Braud et al. [6] suggested a robust and generic method to evaluate the IRIP susceptibility maps using runoff-related proxy data in the railway context. They developed a workflow based on contingency tables (χ² test and false alarm ratios) between areas at risk and observed impacts taking into account hazard levels. However, none of these works could pretend using largely exhaustive and unbiased proxy data on extensive zones to thoroughly evaluate the IRIP method.

The goal of this paper is to use areas impacted by all kinds of extreme rainfall-related disturbances such as gullies, deposits, landslides or mudflows, as proxy data for SWF occurrence. Affected zones were identified using an optical and radar remote sensing-based probabilistic change detection method at 10 m spatial resolution developed in [28,29]. This new repeatable approach was designed specifically to allow for identification of ground deteriorations far from the main hydrographic network. Based on land cadastre plot delineation, land cover mapping [30] and supervised classification methods [31], it allows for the production of maps giving plot-wise probability of damage detection. For this, the machine learning algorithm relies on intra-plot change pixel statistics of vegetation-specific spectral indices. Thanks to these proxy data, this study seeks to comprehensively investigate the quality and limitations of IRIP maps over six watersheds during two extreme Mediterranean events for which the satellite-based method was successfully validated [29]. In order to study the significance of different factors such as land use, pedology or topography on SWF occurrence, the sensitivity to the different IRIP input indicators is also analyzed through contingency statistics and logistic regression, allowing us to suggest improvements to the current methodology. In other works for instance, Niehoff et al. [32] examined the role of land cover conditions on storm-runoff generation using spatially distributed process-based hydrological modeling and found that land-use impacts were specifically relevant during high intensity storms and in small-scale watersheds.

Since the IRIP method only indicates susceptibility to overland flow processes, comparison to actual damages can only be performed if spatially distributed rainfall measurements are available. Indeed, regardless of a site’s susceptibility to runoff, the presence or absence of soil degradations following a given event should be conditional to the actual amount and intensity of rainfall received. Bernet et al. [33] and Spekkers et al. [10] worked on characterizing rainfall events leading to SWF on a country scale (Switzerland for the former and the Netherlands for the latter). They both found that high maximum rainfall intensities over short time windows were a major self-sufficient damage predictor. However, Spekkers et al. [10] emphasized the need to add runoff generating mechanisms as explanatory variables in his models to account for the remaining significant amount of unexplained variance. Therefore, in order for this study to be as comprehensive as possible, analysis of the IRIP intense runoff model is performed considering not only the location of SWF damages but also the spatial distribution of rainfall intensities through radar measurements over the studied events.

To our knowledge, this paper is the first attempt to evaluate a surface runoff susceptibility mapping model using SWF damage proxies derived from satellite remote sensing. Its strength lies in the combination of large scale validated damage maps and rainfall radar measurements to estimate the relevance and suggest improvements to the IRIP method with an operational prospective objective. In addition, the damage proxy maps along with the methodology applied in this paper should be adequate to evaluate the performance and limitations of any other surface runoff mapping model. The paper is organized as follows: Section 2 describes the study areas as well as the multiple materials and methods that were set up in this work. IRIP evaluation results are outlined in Section 3, along with a sensitivity analysis of the model’s input indicators for the production map. The relevance of IRIP’s accumulation factors and precipitation intensity to predict SWF proxy locations is also investigated through logistic regression in that section. Results are discussed in Section 4 while Section 5 presents the concluding statements and opens up on future methodological developments for both remote sensing of SWF and modeling of pluvial runoff.

2. Materials and Methods

2.1. Study Areas

This paper focuses on two extreme flash-flooding Mediterranean events. One of them occurred in the Aude department on 15 October 2018, while the other took place in the mountainous region of Alpes-Maritimes on 2–3 October 2020 (“tempête Alex”). Although landscape and topography greatly differ between these two regions, both episodes induced heavy losses from FFs and SWFs due to high intensity precipitations (up to 250 mm locally in 6 h in Aude [34] and 90 mm in only 1 h at Coursegoules rain station in Alpes-Maritimes [35]). Three watersheds in Aude were selected to be modeled by IRIP over a region of interest of more than 1700 km²: Fresquel, Aude-Sou-Lauquet and Aude-centre. For the Alpes-Maritimes study area, the Tinée, Vésubie and Roya basins, representing 1400 km² of steep mountainous woodlands were examined. Selection of these 6 catchments corresponded to the regions of interest over which identification of SWF damages had already been conducted in [29] (see Section 2.2). In this paper, in relation to the work of Cerbelaud et al. [29], evaluation of the IRIP method relied on plot territorial subdivision. For this, the Aude and Alpes-Maritimes official department land cadastre datasets were retrieved. Characteristics of the study areas with regard to SWF detection are detailed in

Table 1. Characteristics of the study areas depending on satellite detection of SWF damages. Urban areas and plots within 50 m of an active stream were removed from the analysis (Section 2.5).

	Aude		Alpes-Maritimes
	Unaffected	Damaged	Unaffected	Damaged
Number of plots	73,023	12,477	87,758	6591
Total area (km2)	626	191	628	242
Plot size (ha): 25–50–75th	0.14–0.32–0.68	0.36–0.75–1.62	0.03–0.10–0.28	0.09–0.25–0.88
Slope (°): mean (sd)	6.8 (6.8)	3.4 (5.2)	27.4 (12.0)	24.3 (15.7)
Mean—max—sd rainfall over:
15 min (mm)	12–23–4	13–23–4	17–39–7	18–37–7
60 min (mm)	30–62–10	31–62–10	38–107–18	41–107–20
24 h (mm)	128–233–42	129–233–41	209–529–80	234–529–87

(unaffected versus damaged plots, see Section 2.2).

In addition, a map describing the location and topography of all the watersheds in both study sites is provided in Figure 1.

2.2. Sentinel Plot-Based Change Detection (SPCD) Method: SWF Damage Proxy Maps

This study is directly subsequent to the works of Cerbelaud et al. [29]. Because actual intense rainwater runoff leading to SWFs is hardly ever observable, they used an original approach aiming at identifying the related footprints on the ground, especially at a distance from the main hydrographic network. The idea was to associate the plot delineation from the official French land cadastre, land use (LU) data and closely timed images from optical (Sentinel-2) and SAR (Sentinel-1) remote sensing to develop a repeatable plot-based change detection method. Identification of damaged areas at 10 m spatial resolution relied on using a machine learning (ML) algorithm, namely the Gaussian Process Classifier (GPC [31]). Explanatory variables consisted of plot-wise statistics depicting various spatial patterns measured from the temporal variations of vegetation-specific spectral indices. The model was run with a single training sample from one study site, while validation was performed on three distinct events in contrasted environments or at a different season. For this, numerous ground truths confirmed by photo-interpretation of very high resolution (VHR) optical imagery (Pléiades satellite, IGN orthophotos) were collected. Interested readers will refer to [11,28,29] for complete details about the method and the associated results.

In this work, the maps produced by this satellite-based method (hereafter referred to as SPCD method, for “Sentinel Plot-based Change Detection” method) will be considered as proxy data of SWFs for the evaluation of IRIP susceptibility maps (Figure 2). Although damage identification of ground truths was not completely exhaustive on the Aude and Alpes-Maritimes validation samples (producer accuracies from 70% to 85%), false discovery rates (or commission errors) inferior to 9% partly ensured that most areas detected by the SPCD method could be considered with great confidence as proxy data for IRIP maps. Moreover, the very large areas covered (more than 1000 km² for both study sites) could reasonably guarantee consistent and significant evaluation statistics.

2.3. The IRIP Susceptibility Mapping Method

The present description of the IRIP model relies mostly on the original methodological paper from Dehotin and Breil [12]. It is inspired by [6,24,25], but with up-to-date improvements especially concerning the incorporation of rainfall climatological data in the methodology. The IRIP method was designed to represent all the processes related to intense surface runoff occurrence from (i) the generation to (ii) the transfer and (iii) the accumulation of rainwater flow anywhere on a given territory. It is a geomatics method that performs generic mapping at the basin scale without any explicit hydrological modeling. It provides distributed information on the three processes associated with SWF hazards by defining susceptibility levels from 0 (no risk) to 5 (very high risk). For this, 5 binary indicators (=1 if favorable to runoff, =0 otherwise) are computed and summed up for each of the three susceptibility maps, using four types of input data commonly found in hydrological modeling. As the IRIP model focuses on SWF hazards occurring outside the river network, it is complementary to FF risk mapping. First of all, in order to initialize IRIP, a Digital Elevation Model (DEM) of both study sites (IGN BD ALTI^® 25 m here) needs to be processed to determine watershed boundaries and flow directions. The final spatial resolution of the IRIP maps is constrained by that of the DEM.

2.3.1. Production Map

Overland flow of rainwater is more likely to arise over poorly infiltrating surfaces, shallow or saturated soils, and anywhere with little topography. These conditions can result in runoff buildup by infiltration excess and/or saturation excess following heavy rainfall [36]. Land use (LU) data allows discriminating impervious bare surfaces, prone to the production of surface runoff, from infiltrating and rough surfaces that can absorb rainwater and slow down overland flow. Here, each of the 23 classes of the OSO French land cover product [30] available at a 10 m spatial resolution are designated as prone to surface runoff (=1: urban and agricultural, bare grounds mostly) or unfavorable (=0: forests, woody heaths, grasslands) to define a first binary indicator (Figure 3a).

Three other indicators for the production map are derived using soil type (ST) information from the European Soil Data Centre (ESDAC) [38]: slacking rate, permeability and saturated water content. Within the ESDAC database, clay, sand and silt proportions (%) for soil texture are available at 500 m resolution for all of Europe. Cerdan et al. [37] methodology is then used to determine slacking (and erodibility for the transfer map, see next subsection) rates using soil texture triangles. Hydrodynamic characteristics of the top 30 cm soil layer such as saturated hydraulic conductivity (cm.day⁻¹) and saturated water content (cm³.cm⁻³) can be directly downloaded at 250 m resolution through ESDAC (thanks to hydraulic pedotransfer functions developed by Toth et al. [39]). In turn, these two attributes are faced respectively with the 1-h (for infiltration excess) and 24-h (saturation excess) mean rainfall intensities corresponding to a 30-year return period for each study area. This period was selected because most substantial disorders seem to start appearing on hillsides at such a recurrence interval. For this, a large 10-year time series (2006–2016) of quantitative precipitation estimations from Panthere radar-based gridded product (see Section 2.4 for more details) was collected over France at a 5-min time step and 1 km² spatial resolution.

Furthermore, the DEM is processed to consider a combination of the local slope and Beven topographic index [40,41] as a 5th indicator related to topography (Figure 3a). For these two variables, a k-means clustering algorithm is applied making use of both the iterative minimum distance [42] and the hill-climbing [43] methods to derive respective thresholds δ_s and δ_t and divide the grid values into two classes (=1 if >δ, i.e., favorable to runoff, and =0 if ≤δ). This last indicator is then assigned a score of 1 if either the slope or the topographic index is favorable, and 0 if both are not. The Beven topographic index is used because it reveals areas prone to buildup of surface water, where both the upslope drained area is high (resulting in large potential for overland flow) and the local slope is small (preventing a quick water discharge). It is defined as follows:

$$TI=\ln \left(\frac{a}{\tan b}\right),$$

(1)

where $a$ stands for the upslope drained area and $tan b$ the local slope in radians. Eventually, urban areas identified in the OSO map are systematically given a favorable value of 1 for all four LU/ST indicators for the computation of the production map.

2.3.2. Transfer and Accumulation Maps

The first input indicator for the two other maps of susceptibility to transfer and accumulation reflects the need for significant uphill runoff production to induce SWFs. Therefore, for each grid cell, the modal value (i.e., the most represented value) of all IRIP production pixels is estimated among the associated uphill drained area. If this value is greater or equal to 3, the first indicator for both the transfer and the accumulation maps is deemed favorable (=1), and unfavorable otherwise (Figure 3b,c).

Runoff transfer occurs in areas where water can build up speed and flow downwards [44]. Consequences are eroded bedrocks and flattened/uprooted vegetation. A soil erodibility indicator is thus introduced by applying Cerdan et al. [37] method (as for slacking).

Runoff accumulation materializes where water can slow down or even get trapped and where flows can gather and intensify to induce SWF damages such as mud and sediment load deposits. Consequently, the Beven topographic index is adopted alone using the same threshold δ_t to determine a 0 or 1 contribution to the accumulation map.

Susceptibility maps to transfer and accumulation of surface runoff are further computed through 3 topography-related indicators (Figure 3b,c): drained area, local slope and break of slope. The drained area indicator reflects the need for important uphill contribution, with a 2.5 ha threshold derived from Lagadec [5]. The local slope and break of slope indicators have opposite conditions for transfer and accumulation. Indeed, steep slopes favor transfer while low slopes facilitate water accretion. The same threshold δ_s was used in both cases. Eventually, convex breaks of slope are deemed favorable to overland flow transfer, as opposed to concave breaks for accumulation.

The final accumulation maps on the six studied watersheds can be seen in Figure 4 (25 m resolution), along with the Sentinel imagery subset chosen for the SPCD proxy maps (Section 2.2 and Figure 2). For each study area, all plots featuring the same maximum IRIP accumulation score X are referenced in the remainder of this paper as “IRIP_X“ plots_.

2.4. Rainfall Radar Data

Quantitative precipitation estimations (QPEs) by Panthere radar-based gridded products from Météo-France were retrieved on 48 h windows around each of the two events (Figure 5, [45]). These rainfall estimates rely on the same basic physical principles as all active radar-based instruments. Looking up from the ground, the goal is to use relationships between signal attenuation mostly in the X- or C-band and the drop-size distribution (DSD) to estimate the latter. The Panthere QPE uses a dual-polarized C-band antenna to estimate rainfall at a 1 km² resolution every 5 min. Since cloud physics and weather radar polarimetry are still vast open research activities, radar signal inversion to estimate precipitations still suffers from various sources of uncertainties and constitutes a very challenging topic in hydrometeorology [46,47]. Nevertheless, radar data was shown to be more suitable than rain gauge stations to distinctly capture and characterize extreme precipitation at fine scales and over short durations [48].

For this work, maximum rainfall amounts were computed over consecutive periods of 5, 15, 30, 60, 180, 360, 720 and 1440 min on the overall 48 h data windows available, for each 1 km² grid cell of the study areas. It allowed capturing the different maximum rainfall intensities experienced by every 1 km² area over a given period of time, even though each grid cell may have experienced them at different times within the event.

2.5. Validation Workflow and Evaluation Metrics

Many methods have been used for the evaluation of flood mapping models using localized flood impacts [6,20,21,24,25]. The vast majority of them are based on the use of contingency tables and the computation of detection rates and false-alarm ratio. For IRIP, all works to date have used buffers around localized impacts (25 or 50 m) to compensate for uncertainties and approximations in flow paths derived from DEM data (1 or 2 pixels of the BD ALTI^® 25 m). In [6], for instance, if at least one IRIP transfer or accumulation pixel with a score ≥ 4 out of 5 was found within the buffer zone around an impact, it was considered as a “hit”. In this study, following the SPCD damage identification operating mode, detection rates and false alarms were defined and measured at the plot scale (see Table 1 for size characteristics). Because of the spatial resolution of Sentinel satellite imagery (10 m), most SWF damages identified by the SPCD method resulted from accumulation mechanisms. Indeed, erosion footprints are on average more elongated (only a few meters wide) and thus often require finer image resolution for identification. Therefore, only IRIP accumulation pixels were considered for evaluation in this work.

When facing IRIP with localized SWF impacts, information on rainfall spatial and temporal distribution is critical to better delineate the evaluation area, as no surface runoff should be observed from little rainfall accumulation or low-intensity precipitation. Given that the events are often very localized in time, the use of distributed products with short time steps (e.g., 5 min) appears also essential [49].

In this work, we thus propose a quantitative comparison between the IRIP accumulation maps and the SPCD-derived SWF damages taking into account the maximum rainfall intensities over short time steps (Figure 6).

Within each cadastre plot of the regions of interest, we selected the maximum IRIP accumulation score (25 × 25 m² pixels) for evaluation. The maximum rainfall intensity measured over 60 consecutive minutes during a 48 h window around the event was determined at its native spatial scale of 1 km². For the SPCD probability maps, a simple 0.5 threshold was set to determine whether the plot belonged to the “damaged” or “unaffected” class. In order to be sure that most damages were due to surface runoff (SWF) and not to river overflowing (FF), a 50 m buffer zone was applied around all active streams using the French BD Carthage 2017 [50]. All plots within this buffer were then removed from the analysis (damaged or not). In addition, urban areas were also removed from the evaluation framework given that SPCD does not perform detection on this particular land cover type (mainly because of its spatial resolution). Other variables used to determine the relative influence of each IRIP input indicator to explain SFW damage location were also aggregated at the plot scale (see Section 3.3 for the associated results).

2.6. Multivariate Logistic Regression

Logistic regression is a simple and very efficient type of machine learning algorithm for categorical classification [51]. It consists in modeling the probability of a discrete outcome Y (binary here), called dependent variable, given a set of input data X = (x₁, …, x_n), called explanatory variables or predictors (Figure 7). For this, the conditional probability of Y being equal to 1 (as opposed to 0), knowing X, is assumed to be given by a sigmoid-type function called logistic function f: t → f(t) (see Figure 7). The regression coefficients β_i are usually estimated using a maximum likelihood (ML) estimation as loss function. A seemingly analogous measure of the R² found in linear regression is McFadden’s pseudo R-squared [52], ρ², which gives a representation of the extent to which the maximized likelihood of the fitted model is close to 1 using the available predictors. It is important to be aware that, empirically, ρ² usually takes much lower values than those found for R² in linear regression, as stated in [53] (p. 306): “while the R² index is a more familiar concept to planner who are experienced in OLS, it is not as well behaved as the ρ² measure, for ML estimation. Those unfamiliar with ρ² should be forewarned that its values tend to be considerably lower than those of the R² index. […]. For example, values of 0.2 to 0.4 for ρ² represent excellent fit.”.

Results are commonly discussed through the computation of odds ratios (ORs). Depending on the nature of predictors, they need to be computed by taking into account the range over which they roughly evolve. In the Results Section 3, OR represent, for each explanatory variable, the change in the odds of Y being equal to 1 (as opposed to 0) when the predictor varies between its 25th and 75th percentile values (Δ_x variation in Figure 7).

3. Results

3.1. Evaluation of IRIP Maps through Rainfall Estimation and Damage Identification

Spatial statistics and contingency graphs were produced following the evaluation framework on the two studied events and are summarized in Figure 8. Firstly, 52% of the SPCD-identified damaged plots during the Aude event featured an IRIP accumulation score of at least 4 out of 5 compared to 26% for the rest of the unaffected area. These ratios amounted to 41% versus 13% for Alpes-Maritimes (Figure 8, left panel). Considering the number of plots involved (slightly less than 100,000 for each event), IRIP score differences between damaged and unaffected areas were extremely significant (χ² test, p < 0.0001). In addition, for the latter event, 75% of damaged plots were tagged with a 3+ score, as opposed to 38% for the remaining lands. As a consequence, these first simple results reveal that IRIP scores do provide a fairly good indication towards susceptibility to SWF occurrence. Nevertheless, IRIP clearly does not give a perfectly discriminating breakdown of affected areas. The Aude region in particular seemed to be overly tagged at medium or high risk of intense overland flow occurrence (IRIP scores ≥ 3 for more than 70% of the unaffected plots) in the current IRIP framework, especially regarding level 3. This calls for improvements to the methodology to better discriminate certain susceptibility levels, for instance with a more extended score scale (e.g., from 1 to 10).

Secondly, once again with statistics performed over a great number of observations, a clear and significant segregation appeared in terms of damage proportion among the various IRIP accumulation scores (Figure 8, right panel). For the Aude (Alpes-Maritimes) event, 13% (9%), 25% (17%) and 56% (55%) of respectively all IRIP₃, IRIP₄ and IRIP₅ plots were detected as damaged by the SPCD method. Moreover, these proportions increased for both events when considering the areas that experienced higher rainfall intensities. Indeed, the proportion of damaged IRIP₅ (IRIP₄) plots was found to increase up 20% (10%) for the highest quantiles of precipitation (Figure 8, right panel). For both events, following a rather flat and stable part, the 35–40 mm.h⁻¹ threshold seemed to trigger an increase in the proportion of damaged plots among those indicated at risk by IRIP. Interestingly, this threshold corresponds to a saturated hydraulic conductivity of around 1.10⁻⁵ m.s⁻¹, exceeding most commonly found values except for sandy soils [54]. Such rainfall intensity also corresponds to between 5 and 10-year return periods for Aude and Alpes-Maritimes regions [45]. More interpretations are provided in Section 4.1 and Section 4.2. Consequently, IRIP’s score methodology can be confirmed as relevant to define susceptibility levels towards intense surface runoff occurrence.

Eventually, a negative relationship between the mean IRIP accumulation scores and rainfall intensities was found over damaged plots (Figure 9). This tendency was present for all time periods of rainfall measurement, but was the strongest for the lowest ones (the 5, 15 and 30-min precipitation intensities). Therefore, SWFs rather occur over more susceptible areas when rainfall is lower. This confirms that the higher the intensity of rainfall, especially during short time windows, the more areas should be prone to intense overland flow whatever their susceptibility, and thus the less high IRIP scores should be relevant to predict SWF occurrence.

3.2. Sensitivity Analysis of IRIP through Land Use and Pedological Characteristics Used for the Production Map

Land use (LU) and pedological characteristics (ST) are crucial factors controlling the generation of overland flow following heavy rainfall. In this section, a sensitivity analysis of IRIP final evaluation scores is performed on the 4 LU/ST-related input binary indicators for the IRIP production map (Figure 3a): land cover, permeability, saturated water content and slacking crust. For this, each of these indicators was removed from the computation of the production map, leading to lower final scores for the accumulation map.

Land cover and permeability were found to be the highest contributors to overall IRIP accumulation scores (

Table 2. IRIP detection rate (number of IRIP_≥4 plots among SPCD-identified damaged plots), ratio of overall IRIP_≥4 plots and false alarm rate on IRIP_≥5 plots on both study events in the default framework. Differences measured on these ratios are given in the following rows, depending on which indicator was removed from the computation of the production map.

	Aude			Alpes-Maritimes
	Detection Rate (%)	Ratio of IRIP≥4 Plots Overall (%)	False Alarm on IRIP≥5 Plots 1 (%)	Detection Rate (%)	Ratio of IRIP≥4 Plots Overall (%)	False Alarm on IRIP≥5 Plots 1 (%)
Default framework: All indicators	52%	29%	44% (26%)	41%	15%	45% (26%)
No land use	−7%	−6%	−1% (−10%)	−12%	−9%	+2% (+2%)
No permeability	−7%	−6%	−2% (−14%)	−12%	−9%	−5% (−3%)
No (land use + permeability)	−30%	−19%	−0% (−2%)	−16%	−10%	−5% (−1%)
No saturated water content	−2%	−2%	+0% (+6%)	−1%	−1%	+0% (+3%)
No slacking crust	−0%	−0%	−1% (−2%)	−3%	−1%	−2% (+2%)

¹ Numbers in parentheses for the false alarm rate refer to the cases for which a minimum threshold of maximum rainfall intensity (90th percentile) was considered for contingency statistics.

, middle columns). The detection rate, defined as the % of IRIP_≥4 plots within all SPCD-identified damaged plots, was particularly lower when these two indicators were removed from the production score board (Table 2, left columns). On the contrary, the saturated water content and slacking crust contributed little to the final evaluation scores. This is consistent with the flash-flood dimension of the studied events, where Hortonian runoff is expected to play a major role, along with land use characteristics [32]. The false alarm rate on IRIP_≥5 plots, i.e., the % of unaffected plots within all IRIP_≥5 plots, was around 45% for both events, and down to around 25% when considering the areas that experienced the most precipitations. Oddly, false alarm rates decreased when removing certain indicators, in some cases dramatically (e.g., −14% on Aude with no permeability in high rainfall plots) pointing out that these factors were responsible for wrongly indicating high susceptibility in areas that were left unaffected. The negative effects associated with the removal of the most contributing indicators, i.e., land use and permeability, were found to add themselves up and even be amplified (as in Aude) or overlap (Alpes-Maritimes) when removed together. No systematic conclusion could thus be drawn from it. These outcomes call for a more thorough sensitivity analysis of IRIP regarding production factors. Since IRIP currently only considers a binary production contribution depending on a score threshold of 3 out of 5, a better discrimination of the influence of this factor on the final accumulation map should be implemented (see Section 4.3 for suggestions).

3.3. Relevance of Input Indicators for IRIP Accumulation Maps and Rainfall Estimates to Explain SPCD-Derived SWF Locations Using Multivariate Logistic Regression

In parallel to IRIP’s formal evaluation, a multivariate logistic regression was run to try to explain the location of damaged plots as detected by the SPCD method. To this end, the input indicators for the IRIP accumulation map were used before being binarized. Prior to this, correlation statistics were performed on all plots within each study site to avoid collinearity issues within the predictors. The drained area was found correlated (>0.4, p-value < 0.001, not shown) with the Beven topographic index and was thus dropped from the set of explanatory variables. Since the model seeks to explain the location of deteriorations that were event-specific and over a large region, rainfall distribution needed to be added among the predictors so as to determine the genuine influence of each IRIP indicator towards SWF occurrence. The list of input variables for the logistic regression was thus eventually composed of:

i.

Modal value for IRIP uphill production score, otherwise called mode (from 0 to 5, depending mostly on land cover and soil type, see Figure 3a);

ii.

Slope in radians (roughly from 0 to 30° in Aude and to 70° in Alpes-Maritimes);

iii.

Break of slope (convex, flat or concave);

iv.

Beven topographic index (ranges roughly from 0 to around 25);

v.

Maximum rainfall intensity measured over 60 consecutive minutes during a 48 h window around the event.

As the multivariate logistic regression was performed at a plot scale, plot-specific predictors needed to be computed. They consisted of the maximum pixel value contained inside each plot, except for the break of slope for which the presence of a concave break was simply searched for.

Results are displayed in Figure 10. Pseudo R squared values reached 0.12 for Aude and 0.16 for Alpes-Maritimes, which indicated fair to good fit of the model [53]. Substantial conclusions can be drawn from the odds ratios. An OR greater than 1 implies that the odds of a plot being damaged increases when the predictor varies between its 25th and 75th percentile values. Despite the differences in landscape and topography between the two sites, odds ratios were very similar for the Beven topographic index (around 2.20), the modal value for IRIP uphill production score (1.95) and the maximum rainfall intensity (1.25).

Interestingly, the local slope index behaved in opposite manners at the two study sites. For Aude, low local slopes were associated with a greater probability of SWF damage occurrence, as opposed to high slopes for Alpes-Maritimes. This is highly relevant as to the nature of the deteriorations detected in the two sites. For the Aude floods of 2018, most were due to water accretion in low areas within agricultural plots. On the contrary, during the Alex storm in Alpes-Maritimes, a great number of SWFs were observed through landslides and mudflows on steep mountain slopes. Either way, the odds ratios for the maximum local slope predictor were rather low, especially in Alpes-Maritimes (1.05), which pointed to a smaller weight among the overall influence of topography and rainfall factors for intense overland flow occurrence. Eventually, the presence of concave breaks of slope was found either non-significant (p-value > 0.1 on Aude) or with a fairly low odds ratio (19% increase in odds on Alpes-Maritimes).

Examples of SWF damages identified by the SPCD method are displayed in Figure 11 (Aude) and Figure 12 (Alpes-Maritimes). In Figure 11, the plots received a maximum 46 mm of rainfall within 60 min during the event. Gullies can be observed downhill from high local slopes and within a water pathway materialized by flow direction arrows and high values of the Beven topographic index. Furthermore, IRIP uphill production mode is close to its maximum value of 5. The resulting logit score (probability of damage p) from the fitted model is 0.89, indicating the likely presence of an identified SWF, as confirmed by the SPCD damage map. In Figure 12, a mudflow detected by the SPCD method in Alpes-Maritimes was similarly found a high logit score of 0.78, in relation with a 41 mm.h⁻¹ rainfall intensity and incriminating topographical and runoff production factors.

Overall, the conclusions from the logistic regression pointed to a larger influence of the Beven topographic index and the IRIP uphill production mode in explaining the location of SWF damages. Consequently, greater weights and larger segmentation should be attributed to these indicators in comparison with the slope and break of slope factors (see Section 4.3).

4. Discussion

4.1. IRIP SWF Evaluation Results

SWF inundation models have usually featured rather low performances when faced with localized impacts. Indeed, countless factors have to be taken into account simultaneously, making it very hard to implement comprehensive evaluation frameworks [10]. In Bernet et al. [20], four different models, from distributed physics-based to “only” DEM-based, were evaluated at the grid cell level. All models had a low evaluation performance with detection rates around 20–30% for half of the case studies, and around 40–50% for the best models on some other events. Only one event was well captured by all the models (see Figure 6 of [20]). The results were limited due to poor observational data based on an external map that indicated ponded water areas but omitted overland flow paths. The very small areas covered in each case study (~0.5 km²) did not allow either for a good rainfall representativeness with radar measurements being thus constant at the 1 km² scale.

Concerning IRIP, results may differ significantly between pixel-based and buffer-based evaluation methodologies. But there is always a trade-off between detection rates and false alarm rates. Lagadec et al. [24] evaluated IRIP maps on the Lézarde catchment (210 km², northern France) using buffers around proxies derived from witness interviews and field expertise. From 41 incidents listed on 21 railway sections from 1995 to 2012, promising probabilities of detection were achieved (73–90%) using IRIP ≥ 4 transfer and accumulation scores. High false alarm ratios were yet obtained (77–92%), which was explained by the presence of hydraulic mitigation structures as well as the lack of data concerning rainfall spatial variability of the associated events. In another work, detection rates between 94 and 100% and false alarm rates from 20 to 26% were reached by comparing IRIP maps to 17 railway sections selected for hydraulic regeneration works (out of 61 sections) on the Bréauté to Fécamp railway line (20 km long) [25]. These great values partly resulted from the limited number of observations combined to a very good field knowledge that allowed to “calibrate” IRIP, using specific input data and parameterizations adapted from the default framework. Braud et al. [6] eventually assessed the IRIP model on an 80 km railway line in Normandy (north of France) using a comprehensive database of runoff-related impacts (59 sections) covering the whole 20th century. To overcome high false alarm ratios, they combined susceptibility levels to vulnerability stakes and mitigation measures in a unique evaluation framework. By doing so, they improved the false alarm ratio from 62% to 28% while keeping a steady detection rate of 86%.

In this work, no adaptation was made to IRIP’s default framework and very generic countrywide data were used for both sites. Fair evaluation scores were obtained (around 50% detection rates with 25% false alarms on high precipitation areas), but on an infinitely more significant and exhaustive database (1000 km² + area with some 90,000 plots for each event) and taking into account rainfall intensity distribution. Furthermore, as SWF evaluation proxies obtained through the SPCD method are derived solely using radiometric information from multispectral images (in addition to not relying on damage claims), they are independent from IRIP’s input factors (which are mostly DEM-based), making this comparison highly meaningful.

Still, rather low results may be explained on IRIP’s part by several rationales: (i) the coarse resolution of IRIP input data (25 m for the DEM, 10 m for land use and 500 m for pedology); (ii) potentially inadequate land use and soil texture susceptibility knowledge; (iii) the presence of mitigation structures such as hedges, terraces and hydraulic networks that are not taken into account in IRIP and (iv) antecedent soil moisture conditions which are overlooked. The very large dataset of SWF occurrences derived from satellite imagery also suffers from uncertainties inherited from the Sentinel plot-based change detection algorithm: (i) land cadastre subdivision and miscellaneous plot sizes; (ii) LU homogeneity hypothesis within plots and (iii) instrument spatial resolution and location accuracy (see [29] discussion for more details). Nevertheless, because IRIP does not aim at exactly predicting SWF damages but rather at giving a broad representation of the areas that are more or less susceptible, all the results of the present study already appear very satisfying.

4.2. Space-Time Distribution of Rainfall and SWF Occurence

In this work, the IRIP method was confirmed as relevant to indicate susceptibility towards surface runoff occurrence considering damage detection by satellite imagery but also rainfall distribution during two extreme events (Figure 8 and Figure 9). The 35–40 mm.h⁻¹ threshold triggering an increase in the proportion of damaged plots among all high IRIP score plots was consistent with the findings of Dehotin et al. [22]. Indeed, from the results of a 1-year field experiment, they found that surface runoff was observed (either from saturation or infiltration excess) in around 40% of measured events when rainfall intensity was lower than 30 mm.h⁻¹, 80% between 30 and 60 mm.h⁻¹, and up to 100% for 60 mm.h⁻¹ and more. However, these observations did not mention whether significant deteriorations of the land surface had resulted from the occurrence of surface runoff. Marra et al. [49] studied the relationship between debris flow and distribution of precipitation in the eastern Italian Alps. They revealed that the event-cumulated rainfall field (from radar and rain gauge network) systematically exhibited a peak corresponding or close to the debris flow initiation points, with an average 33 mm triggering rainfall mean in one hour (see Figure 5 of [49]). Bernet et al. [33] analyzed SWF damage in Switzerland using geolocated insurance claims on a national scale. Time series of precipitation were linked to the location and date of deteriorations over a large and complex region in terms of topography, pedology and geology. With a gridded precipitation product based on both rain gauges and radar data, they showed that a thresholding method based on local quantiles of maximum intensity (90th) and the total sum of event precipitation (98th) allowed for an excellent identification of events leading to significant SWF occurrence. Naturally, a single absolute threshold could not be set for a countrywide prevention strategy.

Considering the return periods associated with the SWF-triggering rainfall intensities (5 to 10 years), the 30-year return period currently used to compute IRIP soil indicators (K_s and θ_sat) could be lowered. Such thresholds derived from radar-based data yet have to be taken with caution. Indeed, if the spatial distributions obtained from radar are generally more satisfactory than from rain gauge interpolation during heavy storms [48], many studies suggest that radars tend to underestimate absolute rainfall peaks compared with rain gauges up to more than 50% [55,56,57,58,59]. Moreover, it is known that peak intensity biases from radar products can also be important when considering low aggregation timescales (one hour or smaller) in heavy rain ([60] see their Figure 9). Eventually, finer spatial distribution of rainfall could be obtained using downscaling techniques (for example deep learning) or kriging methods with rain gauge data. This would allow better grasping the extent to which the total amount, the timing, or the maximum intensity of precipitation influence SWF occurrence.

4.3. Improvement Suggestions to IRIP Accumulation Map Computation

Logistic regression is a very common and practical tool that has already been used in related geosciences studies. It has been achieved in particular for landslide hazard assessment from aerial photographs and field surveys, in order to design classification models [61]. In that work for instance [61], landslide locations were found to be explained by, in descending order of the most influential factors: (i) land use (from Landsat images); (ii) slope; (iii) topographic aspect; (iv) lithology; (v) Normalized Difference Vegetation Index (NDVI, obtained by the authors from SPOT images) and (vi) topographic curvature. Such results can then be used to tune weights for optimal combinations of landscape determinants in models.

It is very important to acknowledge that precipitation intensity location will always remain a high driver of SWF occurrence [10,33] (OR around 1.25 in logistic regression, Figure 10). IRIP is not designed to take it into account (except in terms of climatological likeliness). Therefore, IRIP’s goal is not to give a perfect representation of where SWF damages have materialized or are going to in the future. However, no physics-based distributed hydrological model can pretend doing so exactly either, once again because the distribution of rainfall in space and time can never be known at the fine required scale. Instead, IRIP can serve as a broad benchmark for prevention strategies on a region to country scale. Modifications ought to be tested to match as closely as possible proxy locations available on different large scale events. Consequently, results from general evaluation and sensitivity analysis of IRIP and most of all from logistic regression in Section 3 inspired the following suggestions to the accumulation map. Considering the values of odds ratios, the Beven topographic index could account for around half of the overall score. More importantly, it should be partitioned into more groups to better segregate risk levels. The same goes for the modal value of IRIP uphill production score, which should account for only slightly less than the topographic index. This way, better analysis of the quality of LU/ST information used as input to IRIP could also be made. Therefore, a new suggestion for the IRIP accumulation map would be to lay it on a 0 to 10 score basis, with the Beven index ranging from 0 to 5, the IRIP production mode from 0 to 3 and the slope and break of slope indicators from 0 to 1, unchanged (Figure 13). High susceptibility levels could then be set around a 6 or 7 score for instance, depending on regions and rainfall events.

Eventually, more precise inputs could be used for IRIP. In particular, DEM with finer spatial resolutions could be implemented (like the RGE ALTI^® 5 m or LiDAR products ≤ 1 m), or even be replaced by Digital Surface Models (DSM) computed through VHR satellite stereo acquisitions (Pléiades or else CO3D in the coming years).

5. Conclusions

SWFs caused by extreme overland flow account for almost half of all flood damage claims every year in France, and most likely in many other regions worldwide. Intense runoff predictive mapping models are yet still very limited, mainly because of the lack of observational data to evaluate them. In this context, a satellite-based classification algorithm called SPCD provides reasonably exhaustive proxy data of SWFs on a very large scale. In this work, the IRIP geomatics approach, a simple score-based susceptibility method, was faced with rainfall radar measurements and damage maps derived from the SPCD method during two flash-flood events. A total of 6 watersheds in the Aude and Alpes-Maritimes departments in the South of France were thus investigated using a highly significant proxy database composed of almost 200,000 plots and over more than 2000 km² of rural areas. Given IRIP’s objective of identifying areas at risk of SWFs, this work shows that the susceptibility levels are very relevant on these two distinct study sites. Indeed, the proportions of damaged plots increase accordingly with higher IRIP scores, and even more so when focusing on where precipitations are the heaviest. Results also corroborate that where rainfall is comparatively lower, SWFs occur on average over riskier areas as defined by IRIP. Using a multivariate logistic regression, precipitation intensity distribution is confirmed as an important driver of SWF occurrence. However, the main predictors of SWF damages location turn out to be the Beven topographic index and the IRIP uphill production mode, which calls for improvements in IRIP’s core framework.

Suggested modifications will have to be implemented and tested on various test sites in futures studies. Essentially, we are confident that damage proxy maps derived from satellite imagery, including using the SPCD method, can partly solve the great lack in extensive observational data to evaluate and improve any kind of SWF modeling workflow. This work is a first step in the comprehensive evaluation of the quality and limits of the IRIP model. Although IRIP maps will always be conditional to where intense precipitations hit, they may be used as an operational tool to broadly identify areas that require further refined hydraulic expertise and simulation.

Eventually, as most footprints related to runoff transfer processes such as erosion and gullies are less likely to be identified at the 10 m spatial resolution of Sentinel imagery, IRIP transfer maps were not included in this evaluation framework. This should be addressed in future works by developing a semantic segmentation approach. Using a U-net convolutional neural network architecture based on the fusion of Sentinel-2 bi-temporal and VHR post event data (Pléiades), more acute detection capacity could be expected.