Landslide susceptibility mapping is the first and most important step involved in landslide hazard assessment. The purpose of the present study is to compare three nonlinear approaches for landslide susceptibility mapping and test whether coal mining has a significant impact on landslide occurrence in coal mine areas. Landslide data collected by the Bureau of Land and Resources are represented by the X, Y coordinates of its central point; causative factors were calculated from topographic and geologic maps, as well as satellite imagery. The five-fold cross-validation method was adopted and the landslide/non-landslide datasets were randomly split into a ratio of 80:20. From this, five subsets for 20 times were acquired for training and validating models by GIS Geostatistical analysis methods, and all of the subsets were employed in a spatially balanced sample design. Three landslide models were built using support vector machine (SVM), logistic regression (LR), and artificial neural network (ANN) models by selecting the median of the performance measures. Then, the three fitted models were compared using the area under the receiver operating characteristics (ROC) curves (AUC) and the performance measures. The results show that the prediction accuracies are between 73.43% and 87.45% in the training stage, and 67.16% to 73.13% in the validating stage for the three models. AUCs vary from 0.807 to 0.906 and 0.753 to 0.944 in the two stages, respectively. Additionally, three landslide susceptibility maps were obtained by classifying the range of landslide probabilities into four classes representing low (0–0.02), medium (0.02–0.1), high (0.1–0.85), and very high (0.85–1) probabilities of landslides. For the distributions of landslide and area percentages under different susceptibility standards, the SVM model has more relative balance in the four classes compared to the LR and the ANN models. The result reveals that the SVM model possesses better prediction efficiency than the other two models. Furthermore, the five factors, including lithology, distance from the road, slope angle, elevation, and land-use types, are the most suitable conditioning factors for landslide susceptibility mapping in the study area. The mining disturbance factor has little contribution to all models, because the mining method in this area is underground mining, so the mining depth is too deep to affect the stability of the slopes.