Identification, 3D-Reconstruction, and Classification of Dangerous Road Cracks
- Creating an actual dataset of heavily textural road cracks that may be used to train and test machine learning algorithms.
- Developing a technique for the automatic classification of cracks.
- Devising a method for 3D reconstruction of road cracks.
2. Related Works
3. Proposed Approach
3.1. Image Processing
3.1.1. Grayscale Conversion
3.1.2. Images Segmentation
- Step 1—Binarization: The image from a grayscale scan has two types of information, the inhomogeneous texture of the backgrounds and the cracks which is weakly contrasted with respect to the texture. To delimit these two classes, we adopted the thresholding technique. The latter is used to reduce the information contained in the image to keep only the useful pixels that represent the objects of interest which are the cracks in our case. We propose the Fuzzy C-Means (FCM) thresholding algorithm . It is an iterative classification method that classifies pixels according to C classes. It calculates each time the centers of the classes and generates the membership matrix U of the pixels in these classes.
- Step 2—Labeling of connected components: The second sub-step consists of grouping the pixels obtained at the binarization step in order to remove the noise and find the entire shape of the crack. This phase consists in extracting regions of connected pixels having common properties using the value of each pixel and the interactions with their neighboring.
- Step 3—Filtering by morphological operators: When there are no imperfections in the photos, the black areas are scattered throughout the picture at random. In the image, these areas appear as noise. We apply a morphological closure using the 8 × 8 disk structuring element in order to connect neighboring pieces of cracks. It is noted that the closure operation is expansion followed by erosion. Similarly, we reduced the thickness of the defect by a skeleton search using the “Thin” filter of size 3 × 3 to facilitate the extraction of parameters such as orientation for the characterization phase. Indeed, the skeleton reduces dimensions. Before that, it is essential to fill the small holes in the components in order to avoid noises on the skeleton. An illustration is given in Figure 3.
- Step 4—3D reconstruction: Image processing is an essential step to detect and extract the region of interest. Figure 4 presents an example of 3D crack reconstruction from the 2D processed image. The 3D representation of the image helps us to calculate the depth of the crack which is considered an important primitive. This characteristic is used to determine the type of crack if it is minor, moderate, or severe.
3.2. Region of Interest Detection
3.3.1. Length and Width
3.3.2. Crack Severity
3.3.3. Projection Attributes
3.3.4. Global Attributes in Hough Space
3.3.5. Parameter Normalization
3.4. Machine Learning and Crack Classification
- Due to the number of support vector machines to manage and the number of classifiers to estimate, we opted for the “one against all” classification strategy which allows us to manage a minimal number of classifiers.
- We opted for the Gaussian kernel RBF (radial basis function) as it is the kernel frequently used in the literature and which has demonstrated the best performance in terms of pavement image classification. The kernel parameter was set to 6 (this value was selected experimentally to provide the optimum accuracy and performance).
- The trade-off C is used to fix the trade-off between minimizing the learning error and maximizing the margin. The higher the value of C, the more the capacity of the classifier is optimal. In our case, the value of C has been fixed in a heuristic way. We opted for a value of C = 1000. To accelerate the learning of SVMs and improve their performance, we used the SMO (sequential minimal optimization) method. Indeed, the SMO algorithm segments the initial optimization problem into sub-problems for which we have an analytical solution. Its implementation is easy to implement since it does not require the use of a particular optimization library.
4. Experimental Results
4.2. Execution and Learning Times
- Crack detection time: 998 ms,
- Crack characterization time: 331 ms, item Region classification time: 393 ms (depending on the number of regions),
- Crack classification time: 59 ms,
- Total: 1781 ms (18 s).
- Region learning time: 421 ms (depending on the number of regions),
- Crack learning time: 171 ms.
4.3. Evaluation of the Crack Detection Phase
4.4. Performance of the Crack Classification Phase
- Increasing the size of our database to improve processing results.
- Taking into account a local threshold determined at the level of each region of the image instead of a global threshold applied to the image. This type of thresholding can solve the problem of false detection and thus allows the detection of other types of degradations in addition to cracks.
Data Availability Statement
Conflicts of Interest
|CNN||Convolution Neural Network|
|SVM||Support Vector Machine|
|GPS||Global Positioning System|
|DGPS||Differential Global Positioning System|
|SMO||Sequential Minimal Optimization|
- UN Road Safety Fund. Available online: https://unece.org/about-un-road-safety-funds (accessed on 21 December 2022).
- Elvik, R.; Høye, A.; Vaa, T.; Sørensen, M. The Handbook of Road Safety Measures, 2nd ed.; Emerald: Bingley, UK, 2014. [Google Scholar]
- Al-Tit, A.A.; Dhaou, I.B.; Albejaidi, F.M.; Alshitawi, M.S. Traffic Safety Factors in the Qassim Region of Saudi Arabia. SAGE Open 2020, 10, 2158244020919500. [Google Scholar] [CrossRef]
- Mannering, F.L.; Washburn, S.S. Principles of Highway Engineering and Traffic Analysis; Wiley: Hoboken, NJ, USA, 2020. [Google Scholar]
- Sambito, M.; Severino, A.; Freni, G.; Neduzha, L. A systematic review of the hydrological, environmental and durability performance of permeable pavement systems. Sustainability 2021, 13, 4509. [Google Scholar] [CrossRef]
- Hamdi, S.; Sghaier, S.; Faiedh, H.; Souani, C. Robust pedestrian detection for driver assistance systems using machine learning. Int. J. Veh. Des. 2020, 83, 140–171. [Google Scholar] [CrossRef]
- Farhat, W.; Sghaier, S.; Faiedh, H.; Souani, C. Design of efficient embedded system for road sign recognition. J. Ambient. Intell. Humaniz. Comput. 2019, 10, 491–507. [Google Scholar] [CrossRef]
- Arya, D.; Maeda, H.; Ghosh, S.K.; Toshniwal, D.; Sekimoto, Y. RDD2020: An annotated image dataset for automatic road damage detection using deep learning. Data Brief 2021, 36, 107133. [Google Scholar] [CrossRef] [PubMed]
- Li, Y.; Ma, J.; Zhao, Z.; Shi, G. A Novel Approach for UAV Image Crack Detection. Sensors 2022, 22, 3305. [Google Scholar] [CrossRef] [PubMed]
- Munawar, H.S.; Hammad, A.W.A.; Haddad, A.; Soares, C.A.P.; Waller, S.T. Image-Based Crack Detection Methods: A Review. Infrastructures 2021, 6, 115. [Google Scholar] [CrossRef]
- Zou, Q.; Cao, Y.; Li, Q.; Mao, Q.; Wang, S. CrackTree: Automatic crack detection from pavement images. Pattern Recognit. Lett. 2012, 33, 227–238. [Google Scholar] [CrossRef]
- Hu, Y.; Zhao, C.X. A novel LBP based methods for Pavement Crack Detection. J. Pattern Recognit. Res. 2010, 5, 140–147. [Google Scholar] [CrossRef]
- Yun, H.B.; Mokhtari, S.; Wu, L. Crack Recognition and Segmentation Using Morphological Image-Processing Techniques for Flexible Pavements. Transp. Res. Rec. 2015, 2523, 115–124. [Google Scholar] [CrossRef]
- Cubero-Fernandez, A.; Rodriguez-Lozano, F.J.; Villatoro, R.; Olivares, J.; Palomares, J.M. Efficient Pavement Crack Detection and classification. EURASIP J. Image Video Process. 2017, 2017, 39. [Google Scholar] [CrossRef][Green Version]
- Pouyanfar, S.; Sadiq, S.; Yan, Y.; Tian, H.; Tao, Y.; Reyes, M.P.; Shyu, M.L.; Chen, S.C.; Iyengar, S.S. A Survey on Deep Learning: Algorithms, Techniques, and Applications. ACM Comput. Surv. 2018, 51, 1–36. [Google Scholar] [CrossRef]
- Mandal, V.; Uong, L.; Adu-Gyamfi, Y. Automated Road Crack Detection Using Deep Convolutional Neural Networks. In Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA, 10–13 December 2018; pp. 5212–5215. [Google Scholar] [CrossRef]
- Chun, P.J.; Yamane, T.; Tsuzuki, Y. Automatic Detection of Cracks in Asphalt Pavement Using Deep Learning to Overcome Weaknesses in Images and GIS Visualization. Appl. Sci. 2021, 11, 892. [Google Scholar] [CrossRef]
- Nguyen, N.H.T.; Perry, S.; Bone, D.; Le, H.T.; Nguyen, T.T. Two-stage convolutional neural network for road crack detection and segmentation. Expert Syst. Appl. 2021, 186, 115718. [Google Scholar] [CrossRef]
- Samma, H.; Suandi, S.A.; Ismail, N.A.; Sulaiman, S.; Ping, L.L. Evolving Pre-Trained CNN Using Two-Layers Optimizer for Road Damage Detection From Drone Images. IEEE Access 2021, 9, 158215–158226. [Google Scholar] [CrossRef]
- Hammouch, W.; Chouiekh, C.; Khaissidi, G.; Mrabti, M. Crack Detection and Classification in Moroccan Pavement Using Convolutional Neural Network. Infrastructures 2022, 7, 152. [Google Scholar] [CrossRef]
- Nayak, J.; Naik, B.; Behera, H.S. Fuzzy C-Means (FCM) Clustering Algorithm: A Decade Review from 2000 to 2014. In Proceedings of the Computational Intelligence in Data Mining, Bhubaneswar, India, 5–6 December 2015; Jain, L.C., Behera, H.S., Mandal, J.K., Mohapatra, D.P., Eds.; Springer: New Delhi, India, 2015; Volume 2, pp. 133–149. [Google Scholar]
- Mukhopadhyay, P.; Chaudhuri, B.B. A survey of Hough Transform. Pattern Recognit. 2015, 48, 993–1010. [Google Scholar] [CrossRef]
- Chandra, M.A.; Bedi, S. Survey on SVM and their application in image classification. Int. J. Inf. Technol. 2021, 13, 1–11. [Google Scholar] [CrossRef]
|Images without cracks||82|
|Images with transversal cracks||86|
|Images with longitudinal cracks||85|
|Images with cracking||60|
|Images with diverse cracks||17|
|Dataset||Rate of Crack Detection||Rate of False Detection|
|Images with crack||79%||21%|
|Image without crack||11%||89%|
|False Detection percentage||16%|
|Dataset||Percentages of Images|
Classified with Cracks
|Percentages of Images |
Classified without Cracks
|Images with crack||98.4%||1.6%|
|Images without crack||7.32%||92.68%|
|Class of the Image||Number of Images||Error||Percentage of Images|
|Percentage of Images|
|1: Without crack||41||3||92.7%||7.3%|
|2: Transversal crack||42||4||90.5%||9.5%|
|3: Longitudinal crack||44||8||81.8%||18.2%|
|5: Other types||9||4||55.6%||44.4%|
|1 (Without Crack)||92.68||2.44||2.44||2.44||0.00|
|2 (Transversal Crack)||0.00||90.48||2.38||7.14||0.00|
|3 (Longitudinal Crack)||2.27||6.82||81.82||9.09||0.00|
|5 (Other Types)||0.00||22.22||22.22||11.11||44.44|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sghaier, S.; Krichen, M.; Ben Dhaou, I.; Elmannai, H.; Alkanhel, R. Identification, 3D-Reconstruction, and Classification of Dangerous Road Cracks. Sensors 2023, 23, 3578. https://doi.org/10.3390/s23073578
Sghaier S, Krichen M, Ben Dhaou I, Elmannai H, Alkanhel R. Identification, 3D-Reconstruction, and Classification of Dangerous Road Cracks. Sensors. 2023; 23(7):3578. https://doi.org/10.3390/s23073578Chicago/Turabian Style
Sghaier, Souhir, Moez Krichen, Imed Ben Dhaou, Hela Elmannai, and Reem Alkanhel. 2023. "Identification, 3D-Reconstruction, and Classification of Dangerous Road Cracks" Sensors 23, no. 7: 3578. https://doi.org/10.3390/s23073578