Landslide Susceptibility Mapping Using DIvisive ANAlysis (DIANA) and RObust Clustering Using linKs (ROCK) Algorithms, and Comparison of Their Performance

Abstract

Landslide susceptibility mapping (LSM) studies provide essential information that helps various authorities in managing landslide-susceptible areas. This study aimed at applying and comparing the performance of DIvisive ANAlysis (DIANA) and RObust Clustering using linKs (ROCK) algorithms for LSM in the Baota District, China. These methods can be applied when the data has no labels and when there is insufficient inventory data. First, based on historical records, survey reports, and previous studies, 293 landslides were mapped in the study area and 7 landslide-influencing attributes were selected for modeling. Second, the methods were clustered in the study area mapping units into 469 and 476 subsets, respectively; for mapping landslide susceptibility, the subsets were classified into 5 susceptibility levels through the K-means method using landslide densities and attribute values. Then, their performances were assessed and compared using statistical metrics and the receiver operating curve (ROC). The outcomes indicated that similarity measures influenced the accuracy and the predictive power of these clustering models. In particular, when using a link-based similarity measure, the ROCK performed better with overall performance accuracy of 0.8933 and an area under the curve (AUC) of 0.875. The maps constructed from the models can be useful in landslide assessment, prevention, and mitigation strategies in the study area, especially for areas classified with higher susceptibility levels. Moreover, this comparison provides a new perspective in the selection of a considerable model for LSM in the Baota District.

1. Introduction

Landslides are among the vast, frequent, and common natural calamities in China [1,2,3,4,5]. One-third of landslide events occurring in China every year occur in the Loess Plateau (which covers about 400,000 square km of land in China), which has a complex geo-environmental nature [6,7,8,9]. These landslides always result in the loss of lives and massive damages, as well as the destruction of roads and railways [10,11,12] (examples are shown in Figure 1). The risk (expected losses or damages) of landslide events will elevate as a result of global climate changes and increasing urbanization caused by the growing population. For this reason, it is crucial to find urgent assessment strategies to prevent and manage landslides and mitigate their consequences.

Landslide susceptibility mapping (LSM) is essential and a common procedure for landslide assessment. Landslide susceptibility refers to the probability of a landslide occurring in an area based on local environmental conditions [13,14]. Typically, LSM is used to predict and map where future landslides may occur [15,16]. Generally, it involves these stages: A collection of spatial data and selection of landslide-related attributes (factors); application of various methods to develop the LSM model, which processes and analyzes the influence of landslide-related attributes on the spatial distribution of landslides; the construction of a landslide susceptibility map; model evaluation. The maps developed in the process can provide different authorities with insights for landslide-susceptible areas for land assessment and management decisions [17].

Several methods have been applied for LSM modeling, including direct mapping, deterministic methods, heuristic methods, probabilistic methods, and machine learning methods (MLM) [18,19,20,21]. Among them, MLM has received increased popularity over the past decade. MLM can generally be categorized as supervised learning and unsupervised learning [22,23]. Supervised learning takes a known set of input datasets (the training dataset) and known class labels of the dataset (the output) and forms a model to predict the class labels of the new inputs. Supervised learning methods are widely applied in LSM modeling in different regions around the world [15,24,25,26,27,28]. Some of the popular supervised learning methods include Decision Tree (DT) [15,29], Logistic Regression (LR) [30,31], Support Vector Machine (SMV) [25,32], Random Forest (RF) [24,33], Artificial Neural Networks (ANN) [34,35], and Naïve Bayes (NB) [26,36]. Despite their popularity, their application becomes limited when there is insufficient data or when the dataset has no labels. To make use of the available and unlabeled dataset, unsupervised learning can be applied [22].

Clustering is a type of unsupervised learning that groups a dataset into subsets called clusters without any training, which makes overall susceptibility mapping possible even when the available dataset is insufficient or when the dataset is large and difficult or expensive to label [37,38,39,40]. The clustering for susceptibility mapping is based on the assumption that mapping units with the same susceptibility have similar values of landslide-influencing attributes. Generally, clustering can be categorized into partition-based methods, hierarchical methods, density-based methods, and model-based clustering methods. Based on these categories, various clustering methods have been proposed for conducting LSM modeling [41,42,43,44,45,46,47]. However, from the literature review, it can be noted that these methods are rare in the field of LSM compared to supervised learning-based methods; at present, there is no agreement on the most suitable method for LSM [48,49,50]. Thus, on this basis and in light of the advantages of clustering methods, there is still a need for more applications and comparative studies among different clustering methods to assess the effectiveness of the models and improve landslide susceptibility assessments.

For this reason, this study presented a comprehensive evaluation of the performance of two hierarchical clustering methods, namely the Divisive ANAlysis (DIANA) and Robust Clustering using linKs (ROCK) [51,52,53,54] methods for LSM in the Baota District, Shaanxi Province, China. These methods have been frequently applied in other fields, including networking, bioinformatics, GPS data, road network construction, and gene expression [55,56,57,58]. The study explored the potential prediction abilities of these clustering algorithms and provided some valuable insight for future research and assessments in LSM, as well as for scientific literature as a whole. The performance evaluation was carried out based on some statistical metrics, namely: sensitivity, specificity, accuracy, kappa, and the receiver operating curve (ROC). Additionally, to show the advantageous capability of clustering algorithms over supervised learning algorithms, the Decision Tree and Random Forest classification methods were employed for comparison based on performance accuracy and ROC.

Moreover, the susceptibility maps generated in this study can recognize and allocate landslide-susceptible places so that land engineers and concerned authorities can decide on favorable places (such as mining areas, new economic zones, and urban areas) for ongoing and forthcoming development arrangements.

2. Overview of the Study Area

The Baota District (Figure 2) is the selected study area. It is found in Yan’an City of Shaanxi Province, which is located in the middle of the Loess Plateau in China. Geographically, it occupies an area of approximately 3556 km² (0.55% of the Loess Plateau coverage) between longitudes 109°14′ E–110°07′ E and longitudes and latitudes 36°11′ N–37°02′ N. Its elevation above sea level ranges between 800 and 1800 m. This area is also recognized as a typical valley area and a fragile environmental area for severe soil erosion and landslides [59]. There are two rivers in the northern and southern parts of the area, the Yanhe River and Fenchuan River, respectively. These features describe the topography of the area. Geomorphological characteristics of the area include gorges and heaved slopes. Additionally, there are deposited rocks and extensive quaternary loess deposits that cover the largest part of the area, explaining the geology of the area [60]. The annual average rainfall is 550 mm, and heavy rainfalls are recorded between June and October, ranging between 58 and 117 mm, which have triggered most of the landslides in the area [44,45,61].

The Baota District serves as Yan’an’s administrative center, where various city government offices are located. It is a significant site for various economic activities such as tourism. However, because of the nature of the environment, prolonged heavy rainfalls, and the growing population leading to the expansion of habitation and the development of social and economic activities in the area, various disasters including landslides tend to occur frequently in the area. In response to this, the assessment and management of these disasters are considerably important [59,62]. Thus, we developed this LSM study, particularly for this area, as a helpful tool in assessing and managing landslides and their impacts.

3. Methodology

The landslide susceptibility mapping is determined by the characteristics of the area, the availability of the data, and the methods used to produce the susceptibility map. It is essential to develop a proper methodological structure that aids in determining susceptibility mapping. For this purpose, this study adopted the following steps: (1) data collection: preparation of landslide database and selection of the influencing attributes based on an extensive literature survey and local geo-environmental conditions, as well as the description of their relationship to landslide occurrence; (2) description of the research methods and their implementation in LSM; (3) a susceptibility map that was constructed based on the obtained results; (4) the evaluation and comparison of the methods’ performance based on internal and external evaluation metrics. The study methodology structure is presented in Figure 3.

3.1. Data Collection

3.1.1. Landslides Database

In LSM studies, the landslide geospatial database is very essential for providing details on past landslides, as well as the relationship between landslides and landslide-influencing attributes [63]. The database used in this study was prepared based on data provided by Xi’an Center for Geological Survey (XCGS). This data was collected through interpretation of the post-disaster aerial photographs and field investigations from 1081 locations in the area, from which 293 historical landslides were recorded (as presented in Figure 2C). The landslides were of two types: rotational and translational, with most of them being rotational landslides. In terms of size, the volume of the sliding mass was classified as: less than 10¹ × 10⁴ m³ (small scale, 30.7% of the recorded landslides); between 10¹ × 10⁴ and 10² × 10⁴ m³, 52.6% of the landslides; between 10² × 10⁴ and 10³ × 10⁴ m³ (large scale, 16.7% of the recorded landslides) [61]. These 293 landslide sites, along with 213 randomly selected non-landslide sites (making 506 samples in total), were applied to evaluate the LSM models in this study. In addition, the database comprised a set of landslide-influencing attributes.

3.1.2. Landslide Influencing Attributes (LIAs)

Selecting LIAs for the development and evaluation of models is a basic phase in LSM [64]. Landslide occurrence can be influenced by various attributes. By referring to previous studies [44,45,46,61] and the availability of data, 7 LIAs were selected: elevation, slope angle, slope aspect, profile curvature, rainfall, lithology, and Normalized Difference Vegetation Index (NDVI). Landslides occur as a result of slope instability [65,66]. Elevation and slope angle play an important role in influencing landslide occurrences because slope instability increases with an increase in elevation and slope angle [67,68]. Landslide events in the Baota District happened in slopes with an elevation range of 20 to 30 m and an angle higher than 60°. The slope aspect describes slope orientation (direction); this regulates the slope’s exposure to sunlight, hydrological processes, and wind direction, which have an impact on the physical characteristics of rock and soil masses and ultimately on slope stability [69,70]. In the study area, landslides are more likely to occur in the northeast direction (between 0°–45° and 315°–360°) [47,61]. Profile curvature describes the slope curving in the downward direction, and its values represent the topography structure. The positive and negative values are indicated as upwardly concave and upwardly convex, respectively, while zero represents a flat surface. In addition, the more strongly negative or strongly positive the value, the higher the possibility of a landslide occurring [71,72]. Lithology is an important attribute that describes the rock and soil structure and properties in an area [73]. NDVI was selected to represent and quantify the vegetation cover of the area. Vegetation cover helps to resist slope movements, reduces soil, and improves slope stability. Thus, barren or sparsely vegetated areas are frequently exposed to landslides [74]. In the study area, most of the landslide events were recorded during and after rainfall periods, indicating that rainfall attribute is directly associated with landslide occurrence [43,45,61]. In addition, several researchers have verified that intense and continuous rainfall affects the slope stability, which consequently results in landslide occurrence [15,29,75,76,77,78,79]. Based on their significant relationship and impact on slope stability, the selected attributes are of great importance in landslide susceptibility assessment, not only in this study area but also in other areas around the world [19,20,80,81,82]. The category, data types, scale/resolutions, class, and sources of the influencing attribute data are shown in

Table 1. Information on the Landslide Influencing Attributes used in this study.

Category	Attribute Name	Data Type	Scale	Class	Data Source
Topography	Elevation (m)	Continuous	1:50,000	0~254	Xi’an Center of Geological Survey
	Slope angle (°)	Continuous		0–6.54, 6.54–13.08, 13.08–18.42, 18.42–22.78, 22.78–26.66, 26.66–30.54, 30.54–34.41, 34.41–39.02, 39.02–61.80
	Slope aspect	Discrete		Flat, North (N), North-East (NE), North-West (NW), East (E), West (W), South-East (SE), South (S), and South-West (SW)
	Profile curvature	Discrete		<−0.05, −0.05 to 0.05, >0.05
Geology	Lithology	Discrete		I: loess + nearly horizontal paleo-soil, II: loess + inclined paleo-soil, III: loess + paleo-soil layers + bedrock, IV: loess + paleo-soil layers + the Neogene clay	Xi’an Center of Geological Survey
Underlying surface	NDVI	Continuous		−0.54~0.99	Xi’an Center of Geological Survey
Triggering attribute	Rainfall (mm)	Uncertain		0~200	Baota Weather Bureau

.

Using ArcGIS 10.2 software, the study area was divided into 5,672,922 mapping units (grids units) with 25 m × 25 m spacing. Each mapping unit was described by the 7 LIAs values. The Digital Elevation Model of the study area was acquired from XCGS, from which the thematic maps for topography and geology attributes were extracted at a scale of 1: 50,000. The NDVI thematic map was constructed using Enhanced Thematic Mapper Plus (ETM+) remote sensing image processing software. Rainfall data was collected based on the meteorological rainfall graphs from Baota Weather Bureau [14]. The thematic maps are presented in Figure 4.

3.2. Research Methods

3.2.1. DIANA Algorithm

DIANA is a hierarchical clustering algorithm that constructs a hierarchy of clusters in the dataset [51,52,83,84]. It works in a top-down manner, meaning that it begins with all points in a single cluster and then splits the cluster into smaller and least similar subclusters. To compute the similarity among the points to be clustered, the algorithm employs the basic Euclidean distance function.

Let $\mathrm{p}=\left({\mathrm{p}}_1,{\mathrm{p}}_2\right)$ and $\mathrm{q}=\left({\mathrm{q}}_1,{\mathrm{q}}_2\right)$ be n-dimensional data points, the Euclidean distance ${\mathrm{d}}_{\mathrm{Euc}}\left(\mathrm{p},\mathrm{q}\right)$ between the points is given by:

$${\mathrm{d}}_{\mathrm{Euc}}\left(\mathrm{p},\mathrm{q}\right)=\sqrt{{\left({\mathrm{q}}_1-{\mathrm{p}}_1\right)}^2+{\left({\mathrm{q}}_2-{\mathrm{p}}_2\right)}^2+\dots +{\left({\mathrm{q}}_{\mathrm{n}}-{\mathrm{p}}_{\mathrm{n}}\right)}^2}$$

(1)

The clusters are split based on the maximum ${\mathrm{d}}_{\mathrm{Euc}}$ (distance between the closest neighboring points in the cluster). The DIANA algorithm is explained in the following steps and illustrated in Figure 5.

1: Load into the algorithm the dataset containing $n$ data points

2: Compute ${\mathrm{d}}_{\mathrm{Euc}}$ between the points and find the maximum ${\mathrm{d}}_{\mathrm{Euc}}$

3: Form a distance matrix using the ${\mathrm{d}}_{\mathrm{Euc}}$ values acquired from the previous step

4: Split the cluster based on the distance matrix into the least similar subclusters

5: From new subclusters, calculate new ${ \mathrm{d}}_{\mathrm{Euc}}$ and update the distance matrix

6: Repeat the process until the points in a cluster are comparatively similar

To its advantage, the DIANA algorithm does not require a pre-defined number of clusters and can detect arbitrarily shaped clusters. However, its main limitation is the use of distance similarity measures, which expose the algorithm to errors and misleading results. This limitation can be avoided using the links approach in the ROCK algorithm.

3.2.2. ROCK Algorithm

ROCK is a hierarchical clustering algorithm that constructs a hierarchy of clusters in the dataset through a bottom-up approach [51,85,86,87]. Initially, it treats every object as a solo cluster and then merges the more similar clusters to form a new cluster. To compute similarity, ROCK uses the links approach instead of distance similarity measures. For a pair of objects, a link is described by the number of common neighbors of the objects. Objects that belong to the same cluster will generally contain a large number of common neighbors, and thus more links, and in case two links do not have any neighbors in common, then their link similarity will be equal to 0. In addition, the objects with no or with few neighbors are regarded as noise and are eliminated. In that manner, merging clusters with the maximum number of links first will lead to the construction of better and more significant clusters.

The implementation of the ROCK algorithm is described in the following steps and illustrated in Figure 6:

• Load the dataset containing $\mathrm{n}$ objects
• Select a random sample of objects from the dataset
• Compute the link value for each pair of objects—that is, the number of shared neighbors between objects
• Perform a bottom-up hierarchical clustering on the data based on the link’s similarity measure
• Compute and employ a goodness measure (Equation (2)) to identify the pair of objects to be merged at each step.

$$\mathrm{G}\left({\mathrm{l}}_{\mathrm{i}},{\mathrm{l}}_{\mathrm{j}}\right)=\frac{\mathrm{link}\left({\mathrm{l}}_{\mathrm{i}},{\mathrm{l}}_{\mathrm{j}}\right)}{{\left({\mathrm{n}}_{\mathrm{i}}+{\mathrm{n}}_{\mathrm{j}}\right)}^{1+2\mathrm{f}\left(\mathsf{\alpha }\right)}-{\mathrm{n}}_{\mathrm{i}}{}^{1+2\mathrm{f}\left(\mathsf{\alpha }\right)}-{\mathrm{n}}_{\mathrm{j}}{}^{1+2\mathrm{f}\left(\mathsf{\alpha }\right)}}$$

(2)

where: $\mathrm{link}\left({\mathrm{l}}_{\mathrm{i}},{\mathrm{l}}_{\mathrm{j}}\right)$ is the number of links between two clusters (say ${\mathrm{c}}_{\mathrm{i}}$ and ${\mathrm{c}}_{\mathrm{j}}$); ${\mathrm{n}}_{\mathrm{i}}$ and ${\mathrm{n}}_{\mathrm{i}}$ stand for the number of objects in the two clusters, respectively. Each object in the links has approximately ${\mathrm{n}}^{\mathrm{f}\left(\mathsf{\alpha }\right)}$ neighbors in the clusters.

6.

Repeat the same procedures and assign the rest of the objects to the clusters that have been created.

Note:

• The pair of clusters with maximum $\mathrm{G}\left({\mathrm{l}}_{\mathrm{i}},{\mathrm{l}}_{\mathrm{j}}\right)$ is considered to be the best pair to be merged.

Unlike many other clustering algorithms, ROCK does not require the user to specify the number of clusters in advance, and can find clusters with varying shapes and sizes. In addition, the link approach supports the successful elimination of noise, and thus can identify better clusters.

3.2.3. Implementation of DIANA and ROCK Clustering Methods in LSM

The implementation of these methods in LSM is based on the assumption that mapping units with the same susceptibility have similar values of influencing attributes [88]. In this study, the values of the 7 LIAs were represented by a band of mapping units. The values of all LIAs at a given unit form a vector (A₁, A₂ …, A₇), which was treated as a point/object (as used in the DIANA and ROCK methods, respectively) in the 7-dimension property space. Then, the mapping units correspond to a point/object set in the space. Therefore, as the mapping unit data were prepared, each unit was normalized by its maximum value so that the value of every attribute is between 0 and 1. Then, the data were used as inputs into the DIANA and ROCK clustering methods. The methods find the units with similar characteristic values and categorize them into various clusters/subsets. All of these operations, including conversion, calculation, and clustering analysis, were facilitated by ArcGIS 10.2 under its effective spatial analyst toolbox and Python programming language.

3.3. Methods for Landslide Susceptibility Classification

Based on the existing LSM literature in the study area, landslide susceptibility is usually classified into 5 levels: very high susceptibility level (VHSL), high susceptibility level (HSL), moderate susceptibility level (MSL), low susceptibility level (LSL), and very low susceptibility level (VLSL) [43,44,46,47,60,89,90]. However, the DIANA and ROCK methods categorized the mapping units (points/objects as used in the methods) to their respective subclasses, but did not identify the susceptibility levels in the subsets. Thus, to classify the obtained subsets into the 5 susceptibility levels, a k-means clustering algorithm and landslide density were applied in this study.

3.3.1. K-Means Algorithm

K-means is a simple clustering algorithm that finds k-groups of points from the unlabeled dataset [41,91,92]. Given a dataset containing several points, the algorithm is described below:

• Define the value of “k”.
• Randomly select k-initial centroids.
• Assign each point to its nearest centroid to form a group.
• Calculate and update the centroid (mean) of each group.
• Repeat steps 3–4 until no point changes the group.

To implement this algorithm, the subsets obtained from the DIANA and ROCK clustering methods are treated as points in the K-means algorithm, and the value of “k” is set to 5, denoting 5 susceptibility levels. However, the results still present the statistical information about the 5 groups (susceptibility levels) but do not clarity their meanings (as which group belongs to VHSL, HSL, MSL, LSL, or VLSL).

3.3.2. Landslide Density

Landslide density (LDen) was applied to clarify the meaning of the obtained susceptibility levels. It was calculated in ArcGIS 10.2 platform by computing the number of landslides (from the landslide database) per 1 km² of the mapping unit in each subset. The general principle “the higher the landslide density, the higher the susceptibility level” was applied to clarify the susceptibility levels [45,88,93]. Moreover, for LDen = 0, the susceptibility levels were identified by examining the values of LIAs.

3.4. Performance Evaluation and Comparison Methods

3.4.1. Performance Evaluation

Performance evaluation is the process of applying various evaluation metrics to understand how well or how badly the method has performed on the given data [67,94]. Evaluating the performance of a clustering method is still a problematic and controversial issue because there is no universal criterion for evaluation. However, several criteria have been developed in the literature [95,96,97]. These criteria are generally categorized into two: (i) internal criteria, which usually measures the compactness of the clusters using some similarity measure, and do not use any external information other than the data itself; (ii) external criteria, which is useful for examining whether the clustering results match some external information about the data (inventory data) [44,98,99].

To evaluate the performance of the models based on internal criteria, the Silhouette value is selected as the clusters’ evaluation criterion [100,101]. Silhouette value is calculated by the closeness of the data samples and it is expressed using the following equation:

Silhouette value for a point x within a cluster C_i is given by:

$$\mathrm{Sil}\left(\mathrm{x}\right)=\frac{\mathrm{v}\left(\mathrm{x}\right)-\mathrm{u}\left(\mathrm{x}\right)}{\max \left\{\mathrm{v}\left(\mathrm{x}\right),\mathrm{u}\left(\mathrm{x}\right)\right\}}$$

(3)

where u(x) represents the average distance between sample x and other samples in the cluster (x ∈ ${\mathrm{C}}_{\mathrm{i}}$), and v(x) = max {dis(x,C_i)} [44]. The value of the Silhouette value ranges between −1 and 1. The closer the value is to 1, the better the clustering effect [44,102,103].

However, to assess performance in case of comparison, the external criterion is preferred. Here, the general concept that landslide susceptibility modeling is a binary problem is applied [70]. In addition, the prepared dataset containing 506 samples (293 landslide samples and 213 non-landslide samples) is used for performance evaluation. To this end, the confusion matrix is used, and each sample is given a prediction of either positive (landslide) or negative (non-landslide) [44,104,105]. From the matrix, 4 results are produced: true positive ($tp$), false negative (fn), false positives ($fp$), and true negatives ($tn$). Based on these results, sensitivity ($st$), specificity ($sp$), accuracy, and kappa were selected as the model evaluation metrics (Equations (4)–(7), respectively). These metrics are common and widely used in LSM studies [20,28,73,74,106].

$$\mathrm{st}=\mathrm{tp}/\left(\mathrm{tp}+\mathrm{fn}\right)$$

(4)

$$\mathrm{sp}=\mathrm{tn}/\left(\mathrm{fp}+\mathrm{tn}\right)$$

(5)

$$\mathrm{Accuracy}=\left(\mathrm{tp}+\mathrm{tn}\right)/\left(\mathrm{tp}+\mathrm{tn}+\mathrm{fp}+\mathrm{fn}\right)$$

(6)

$$\mathrm{kappa}=\left({\mathrm{P}}_{\mathrm{a}}-{\mathrm{P}}_{\exp }\right)/\left(1-{\mathrm{P}}_{\exp }\right)$$

(7)

whereby: ${\mathrm{P}}_{\mathrm{a}}=\left(\mathrm{tp}+\mathrm{tn}\right)/\left(\mathrm{tp}+\mathrm{tn}+\mathrm{fp}+\mathrm{fn}\right)$ and ${\mathrm{P}}_{\exp }=(\left(\left(\mathrm{tp}+\mathrm{fn}\right)\left(\mathrm{tp}+\mathrm{fp}\right)+\left(\mathrm{tn}+\mathrm{fp}\right)\left(\mathrm{tn}+\mathrm{fn}\right)\right)/\left(\sqrt{\mathrm{tp}+\mathrm{tn}+\mathrm{fp}+\mathrm{fn}}\right)$.

In addition, the ROC (receiver operating characteristic) curve—a quantitative research method—was used to evaluate the models. It is a graph plotted by sensitivity (on the y-axis) that indicates the proportion of correctly predicted landslide samples against 1-specificity (x-axis), which indicates the proportion of incorrectly predicted non-landslide samples [107]. The areas under the ROC curves (AUCs) were applied to judge the prediction performance. The AUC values range between 0.5 to 1, and a higher value indicates greater performance [67,68,70].

3.4.2. Comparison Methods

To further assess their effectiveness, ROCK and DIANA methods were compared with popular supervised learning methods: DT and RF based on performance accuracy and ROC.

DT is applied in classification tasks to classify labeled data and make predictions [29,108,109]. It aims to construct a model that predicts the class of a target sample by learning some simple decision rules drawn from data structures. DT is based on a tree structure that is composed of a root node, a set of internal nodes, and a set of terminal nodes (leaves). Each node makes a binary decision separating one or more classes from the remaining classes; it is executed by moving down the tree until the terminal node is obtained. The DT model is easy to construct and interpret, which makes it easy for decision makers to use, and thus is commonly adopted to assess landslide predictions [29,109,110].

Random Forest is a supervised learning method that consists of multiple decision trees [24,33]. While classifying a sample, the final result is obtained through a voting mechanism of many decision trees. This means that each tree provides a predicted result, and the result with the most votes (selected by most trees) is taken as the optimal output of the RF method. RF can classify a large amount of higher-dimensional data and has a high tolerance for noise. Thus, it is one of the most commonly used classification methods with high prediction accuracy and is very popular in LSM studies [111,112].

To implement the methods in LSM modeling, the dataset containing 506 samples was divided into training and verification sets: 30% of the samples for training and 70% for verification. The procedure was conducted repeatedly by adding 10% of the data from the verification to the training set until the training set had 70% of the total dataset.

4. Results

4.1. Clustering Analysis

Clustering for mapping susceptibility is based on the assumption that mapping units with the same susceptibility features (geology and geomorphology features) have similar values of LIAs. Based on the aforementioned procedures, 469 and 476 distinct and arbitrary subsets were obtained from the DIANA and ROCK clustering models, respectively. Figure 7 portrays the distribution of those subsets in the study area, whereby different subsets are indicated by different colors in the figure. The ability of the models to identify distinguishing features of the mapping units and cluster them into their respective subsets without prior knowledge of the mapping units belonging indicates that the models have good and effective performance capability [46,113].

4.2. Landslide Susceptibility Mapping

In this study, to map landslide susceptibility in the obtained subsets, the k-means clustering algorithm, LDen, and attribute values were applied to classify the subsets into 5 susceptibility levels. The LDen values obtained in both models ranged between 0 and 1.8. Based on the two models, the attribute values, their corresponding $LDen$ values, and susceptibility levels for some subsets are shown in

Table 2. Description of some subsets obtained based on (A) DIANA (B) ROCK models.

(A)
Subset Id.		Attribute Values (Before Normalization)						Landslide Density			Susceptibility Level
	Elevation	Slope Angle	Profile Curvature	Slope Aspect	Lithology	NDVI	Rainfall	Area (km2)	Land-slides	LD (/km2)
1	32.41	26.89	0.026	S	II	0.67	32–286	9.54	8	0.84	HSL
2	25.35	21.67	0.041	SE	IV	0.56	24–237	8.92	0	0	Based on expertise
…	…	…	…	…	…	…	…	…	…	…	…
190	21.88	30.38	0.61	S	III	0.69	38–189	12.34	9	0.73	MSL
…	…	…	…	…	…	…	…	…	…	…	…
(B)
Subset Id.		Attribute Values (Before Normalization)						Landslide Density			Susceptibility Level
	Elevation	Slope Angle	Profile Curvature	Slope Aspect	Lithology	NDVI	Rainfall	Area (km2)	Landslides	LD (/km2)
1	29.89	24.82	0.032	S	II	0.77	30–283	9.53	7	0.73	MSL
2	21.99	19.19	0.043	N	III	0.64	26–232	6.67	5	0.74	MSL
…	…	…	…	…	…	…	…	…	…	…	…
235	14.89	39.43	0.61	NE	II	0.71	20–150	15.32	0	0	Based on expertise
…	…	…	…	…	…	…	…	…	…	…	…

. The distribution of the susceptibility levels obtained in both models is shown in Figure 8. As for the DIANA model, the very high level occupied the largest proportion, at 29% of the study area, while the very low level occupied the smallest proportion, at 11%. High and low levels accounted for 14% and 15%, respectively, and moderate levels accounted for 28%. Compared with the DIANA model, the ROCK model classified the very high and high levels as 33% and 16% of the study area, respectively, which were all more than those of the DIANA model. The moderate, low, and very low levels accounted for 26%, 15%, and 10%, respectively.

4.3. Evaluation and Comparison Results

The performance of the two models was assessed and compared using the Silhouette as an internal evaluation criterion, and statistical metrics and ROC for external evaluation criteria. ROCK and DIANA models obtained Silhouette values of 0.8677 and 0.8543, respectively. For the statistical metrics,

Table 3. Model assessments and comparative results.

Models	DIANA	ROCK
tp	258	280
tn	186	192
fp	27	21
fn	35	33
St	0.8805	0.8874
sp	0.8732	0.9014
Kappa	0.7518	0.7828
Silhouette	0.8543	0.8677
Accuracy	0.8775	0.8933

was prepared to show the performance and comparative results of the models. In grouping the landslide samples, the DIANA and ROCK models obtained st values of 0.8805 and 0.8874, respectively, similarly for the prediction of non-landslide samples; they also obtained sp values of 0.8732 and 0.8732, respectively. Moreover, the two models obtained kappa of 0.7518 and 0.7828, respectively, and accuracy of 0.8775 and 0.8933, respectively. In addition, for the case of ROC (Figure 9), the DIANA and ROCK landslide susceptibility models showed AUC values of 0.854 and 0.875, respectively.

Moreover, the performances of DIANA and ROCK models were compared with RF and DT-supervised learning models. Under ROC, the AUC of RF was a little higher (0.879) than ROCK (0.875), followed by DIANA (0.854), while DT showed the lowest AUC (0.839) (Figure 9). In addition, as shown in Figure 10, the performance of RF and DT kept on increasing with the increase in data samples. For instance, with 30–70% of the sample data, the accuracy values of DT were 0.348, 0.599, 0.664, 0.747, and 0.868, respectively; for RF: 0.468, 0.654, 0.878, 0.898, and 0.906, respectively. With 30–70% of the sample data, ROCK and DIANA showed steady accuracy values of (0.867, 0.869, 0.872, 0.875, and 0.877) and (0.878, 0.881, 0.887, 0.890, and 0.893), respectively.

5. Discussion

Landslides are complicated and disastrous natural disasters that require immediate and appropriate assessment measures. Conducting landslide susceptibility mapping is a primary step in assessing landslides, by which the susceptible areas and non-susceptible areas can be located and assessment measures can be employed. Therefore, this study applied and compared two clustering-unsupervised learning methods, namely DIANA and ROCK, in mapping landslide susceptibility in the Baota District of Yan’an city in Shaanxi, China. Silhouette internal evaluation metrics, as well as sensitivity, specificity, accuracy, and kappa and ROC external evaluation metrics were employed to evaluate and compare their performances. Moreover, to evaluate the efficiency and effectiveness of these methods, DT and RF-supervised methods were applied for comparison.

From the analysis, both DIANA and ROCK models performed very well because their Silhouette values were more than 0.85. This implies that they both have good clustering capability; thus, they can be effective in assessing landslide susceptibility. However, the results indicate that the ROCK model has comparatively better performance than the DIANA model in terms of sensitivity, specificity, accuracy, and kappa metrics. In addition, the overall performance based on the area under the ROC (AUC) was higher for the ROCK model than the DIANA model. The outstanding performance of ROCK was mainly supported by the application of the link similarity measure, which introduces a global approach to the clustering procedure and facilitates the capturing of global information (knowledge) of neighboring mapping units into the relationship between individual pairs of mapping units. Applying this approach in deciding which mapping units to be merged will lead to the construction of better and more significant subsets. In addition, through this approach, the algorithm can detect significant subsets and eliminate noise. These features add value to the ROCK model, thus providing better and more robust performance. The DIANA model is based on the distance similarity measure, which does not reflect the property of the mapping units’ neighborhood. It is a local approach to clustering tasks and is vulnerable to errors because two different subsets may have a few mapping units (which are then considered noise) that could be very close, and distance similarity could merge the two subsets, causing the situation to worsen as clustering continues. This problem is successfully handled using the link approach. With these limitations, the DIANA model’s performance is less than the ROCK model.

Moreover, in comparison with supervised learning models, Random Forest showed the highest accuracy and AUC, followed by ROCK and DIANA. However, like other supervised learning methods, to obtain higher performance, the models demand a large amount of training data and with less data, performance will always be low. This is a limitation because it is a fact that some study areas may not always have sufficient data to make the algorithms perform highly, and the process of getting information from landslide databases and sites is very exhaustive and expensive. In addition, the supervised learning models are sensitive to changes in the training dataset, meaning that slight changes in the training dataset can cause big variations in the process. These limitations suggest that these supervised learning models cannot be reliable in landslide susceptibility mapping. Fortunately, these limitations can be successively avoided using the ROCK and DIANA models unsupervised learning models. Therefore, from this analysis and discussion, it can be concluded that the ROCK and DIANA unsupervised learning models are more advantageous and more resourceful than the supervised learning models. They can be successfully used in mapping landslide susceptibility and other real-world problems.

It is generally known that the predicted landslide samples should appear in very high or high susceptibility areas as much as possible, while non-landslide samples should appear in a safe area with low or very low susceptibility. Moreover, the constructed landslide susceptibility maps indicate areas that are very high and highly susceptible to landslide occurrences. The map constructed based on the ROCK model showed that a large area (33% coverage) along the Yanhe River is very highly susceptible (indicated by red dots in the figures) to landslides, which is more than the prediction (29%) from the DIANA model. Amounts of 16% and 14% of the subsets from the ROCK and DIANA models, respectively, fell in high susceptible levels (indicated by yellow dots), mostly in the upper part of the study area. In addition, the models predicted that 10% and 11% of the subsets, respectively, were very low, while 15% and 18% of the subsets, respectively, were in the low susceptible level, mostly in the southern part of the area. Upon observing and comparing these results and the inventory map shown in Figure 2, it can be observed that: though the ROCK model had better predictions than the DIANA model, predictions from both models were consistent or in close agreement with the landslide database applied for model construction. This information can be very useful to concerned authorities, decision makers, and residents regarding activities going on in the susceptible areas and to be aware of the landslide risk and its consequences; thus, appropriate measures can be taken. Meanwhile, areas predicted at lower susceptibility levels can continue to be protected by following environmental guidelines for safety assurance.

6. Conclusions

The main purpose of this research was to apply and compare the performance of DIANA and ROCK hierarchical clustering methods for mapping landslide susceptibility in the Baota District, one of the landslide-susceptible areas in China. The study also delivers an evaluation regarding the impact of similarity measures in these clustering methods, while pinpointing the most accurate and reliable clustering method. The models were evaluated and compared based on the Silhouette measure, specificity, sensitivity, accuracy, kappa, and ROC. Both models obtained Silhouette values very close to one, implying that they both have good clustering capability, and can be effective in assessing landslide susceptibility. However, the ROCK model performed better than the DIANA model, with accuracy = 0.8933, kappa = 0.7828, and AUC = 0.875. This is because ROCK uses the link similarity measure, which facilitated the construction of better and more significant clustering results, hence obtaining higher and more robust performance, whereas DIANA uses a distance similarity measure that is prone to errors, which then led to the development of low-quality and less significant clusters, hence lower performance. Moreover, it was noted that the similarity measures have a great impact on the clustering results, and it should be a point of concern while applying these models in LSM studies. In addition, the LSM maps constructed based on these models and the knowledge extracted from them may be significant for the determination and implementation of landslide mitigation schemes. However, the rainfall attribute is of an uncertain data type (because it is recorded in intervals), but, like other clustering methods that rely on distance similarity measurements, the DIANA method did not take into consideration the uncertainty of this attribute. Thus, in the future, this study can be improved by using appropriate methods to process the uncertain data. In addition, more internal criteria should be explored to further evaluate the methods’ performance, and data from other study areas should be applied to validate the methods’ robustness, reproducibility, reliability, and objectivity.