# An Algorithm for Crack Detection, Segmentation, and Fractal Dimension Estimation in Low-Light Environments by Fusing FFT and Convolutional Neural Network

^{*}

## Abstract

**:**

## 1. Introduction

- Supplementary crack detection dataset with the open source crack segmentation dataset for network training and testing.
- Designed an automated methodology capable of detecting, segmenting, and estimating the fractal dimension of cracks in an end-to-end manner called DSD-Net.
- Designed a dual encoder structure based on FFT and CNN. The FFT-based target detection module effectively captures crack patterns and enhances crack localization. The CNN-based segmentation module accurately delineates the crack boundary by considering local and global context information.
- Extensive performance evaluation of the proposed method using several evaluation metrics and comparison with mainstream sum detection and segmentation methods.

## 2. Methods

#### 2.1. Crack Detection and Segmentation System

#### 2.1.1. Frequency Domain Encoder

#### 2.1.2. CNN Encoder

#### 2.1.3. Attention Fusion Detection Module

#### 2.1.4. Loss Functions

#### 2.2. Fractal Computing System

Algorithm 1 Calculate Fractal Dimension | |

**Require**:- image_path (image path), min_box_size (minimum box size), max_box_size (maximum box size)
| |

1: | Read the image from image_path and convert it to grayscale. |

2: | If max_box_size is not provided then |

3: | Set max_box_size to half the minimum dimension of the image. |

4: | end if |

5: | Function box_count(box_size): |

6: | Initialize count to 0. |

7: | for each box with size box_size do |

8: | if the box contains a crack then |

9: | Increment count by 1 |

10: |
end if |

11: | end for |

12: | return count |

13: | Function fractal_dimension(): |

14: | Initialize an empty list counts. |

15: | for each box_size from min_box_size to max_box_size do |

16: | Calculate the number of boxes that cover cracks using box_count(box_size) and store it in counts. |

17: | end for |

18: | Fit a line to the pairs of box sizes and counts using the np.polyfit function. |

Calculate the fractal dimension using the fitted line. | |

19: | return Fractal Dimension |

20: | Function caculate_fractal_dimension (image_pathe, min_box_size, max_box_size): |

21: | Convert the image at image_path to grayscale. |

22: | if max_box_size is not provided then |

23: | Set max_box_size to half the minimum dimension of the image. |

24: | end if |

25: | Call fractal_dimension() to calculate the fractal dimension. |

26: | return Fractal Dimension |

## 3. Implementation

#### 3.1. Public Crack Datasets

#### 3.2. Implementation Details

#### 3.3. Evaluation Metrics

## 4. Experiments and Analyses

#### 4.1. Crack Detection Results

#### 4.2. Crack Segmentation Results

#### 4.3. Fractal Dimension Estimate

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Chen, K.; Reichard , G.; Xu, X.; Abiola, A. Automated crack segmentation in close-range building façade inspection images using deep learning techniques. J. Build. Eng.
**2021**, 43, 102913. [Google Scholar] [CrossRef] - Chen, C.; Seo, H.; Jun, C.; Zhao, Y. A potential crack region method to detect crack using image processing of multiple thresholding. Signal Image Video Process.
**2022**, 16, 1673–1681. [Google Scholar] [CrossRef] - Nnolim, U.A. Automated crack segmentation via saturation channel thresholding, area classification and fusion of modified level set segmentation with Canny edge detection. Heliyon
**2020**, 6, e05748. [Google Scholar] [CrossRef] [PubMed] - Wang, G.; Peter, W.T.; Yuan, M. Automatic internal crack detection from a sequence of infrared images with a triple-threshold Canny edge detector. Meas. Sci. Technol.
**2018**, 29, 025403. [Google Scholar] [CrossRef] - Hong, Y.; Lee, S.J.; Yoo, S.B. AugMoCrack: Augmented morphological attention network for weakly supervised crack detection. Electron. Lett.
**2022**, 58, 651–653. [Google Scholar] [CrossRef] - Matlack, G.R.; Horn, A.; Aldo, A.; Walubita, L.F.; Naik, B.; Khoury, I. Measuring surface texture of in-service asphalt pavement: Evaluation of two proposed hand-portable methods. Road Mater. Pavement Des.
**2023**, 24, 592–608. [Google Scholar] [CrossRef] - Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada, 7–12 December 2015; Volume 28. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Sekar, A.; Perumal, V. Automatic road crack detection and classification using multi-tasking faster RCNN. J. Intell. Fuzzy Syst.
**2021**, 41, 6615–6628. [Google Scholar] [CrossRef] - Li, R.; Yuan, Y.; Zhang, W.; Yuan, Y. Unified vision-based methodology for simultaneous concrete defect detection and geolocalization. Comput.-Aided Civ. Infrastruct. Eng.
**2018**, 33, 527–544. [Google Scholar] [CrossRef] - Zhou, Z.; Zhang, J.; Gong, C.; Wu, W. Automatic tunnel lining crack detection via deep learning with generative adversarial network-based data augmentation. Undergr. Space
**2023**, 9, 140–154. [Google Scholar] [CrossRef] - Chu, H.; Wang, W.; Deng, L. Tiny-Crack-Net: A multiscale feature fusion network with attention mechanisms for segmentation of tiny cracks. Comput.-Aided Civ. Infrastruct. Eng.
**2022**, 37, 1914–1931. [Google Scholar] [CrossRef] - Zhang, Y.; Huang, J.; Cai, F. On bridge surface crack detection based on an improved YOLO v3 algorithm. IFAC-PapersOnLine
**2020**, 53, 8205–8210. [Google Scholar] [CrossRef] - Chen, L.; Yao, H.; Fu, J.; Ng, C.T. The classification and localization of crack using lightweight convolutional neural network with CBAM. Eng. Struct.
**2023**, 275, 115291. [Google Scholar] [CrossRef] - Huang, H.w.; Li, Q.t.; Zhang, D.m. Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunn. Undergr. Space Technol.
**2018**, 77, 166–176. [Google Scholar] [CrossRef] - Shang, J.; Xu, J.; Zhang, A.A.; Liu, Y.; Wang, K.C.; Ren, D.; Zhang, H.; Dong, Z.; He, A. Automatic Pixel-level pavement sealed crack detection using Multi-fusion U-Net network. Measurement
**2023**, 208, 112475. [Google Scholar] [CrossRef] - Xiang, C.; Guo, J.; Cao, R.; Deng, L. A crack-segmentation algorithm fusing transformers and convolutional neural networks for complex detection scenarios. Autom. Constr.
**2023**, 152, 104894. [Google Scholar] [CrossRef] - Fan, Z.; Li, C.; Chen, Y.; Di Mascio, P.; Chen, X.; Zhu, G.; Loprencipe, G. Ensemble of deep convolutional neural networks for automatic pavement crack detection and measurement. Coatings
**2020**, 10, 152. [Google Scholar] [CrossRef] - Jang, K.; An, Y.K.; Kim, B.; Cho, S. Automated crack evaluation of a high-rise bridge pier using a ring-type climbing robot. Comput.-Aided Civ. Infrastruct. Eng.
**2021**, 36, 14–29. [Google Scholar] [CrossRef] - Zhao, S.; Zhang, D.; Xue, Y.; Zhou, M.; Huang, H. A deep learning-based approach for refined crack evaluation from shield tunnel lining images. Autom. Constr.
**2021**, 132, 103934. [Google Scholar] [CrossRef] - Xiang, C.; Wang, W.; Deng, L.; Shi, P.; Kong, X. Crack detection algorithm for concrete structures based on super-resolution reconstruction and segmentation network. Autom. Constr.
**2022**, 140, 104346. [Google Scholar] [CrossRef] - Koga, K.; Yasumura, N.; Shigeta, Y.; Shinjin, M.; Nakagawa, K. Examination of TCI for the quantitative integrity of tunnel lining. Proc. Tunn. Eng. JSCE
**2004**, 13, 371–376. [Google Scholar] - Shigeta, Y.; Tobita, T.; Kamemura, K.; Shinji, M.; Yoshitake, I.; Nakagawa, K. Propose of tunnel crack index (TCI) as an evaluation method for lining concrete. Doboku Gakkai Ronbunshuu
**2006**, 62, 628–632. [Google Scholar] [CrossRef] - Zou, Z.; Chen, K.; Shi, Z.; Guo, Y.; Ye, J. Object detection in 20 years: A survey. Proc. IEEE
**2023**, 111, 257–276. [Google Scholar] [CrossRef] - Jiang, Y.; Zhang, X.; Taniguchi, T. Quantitative condition inspection and assessment of tunnel lining. Autom. Constr.
**2019**, 102, 258–269. [Google Scholar] [CrossRef] - Cha, Y.J.; Choi, W.; Büyüköztürk, O. Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng.
**2017**, 32, 361–378. [Google Scholar] [CrossRef] - Arjovsky, M.; Shah, A.; Bengio, Y. Unitary evolution recurrent neural networks. In Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA, 20–22 June 2016; pp. 1120–1128. [Google Scholar]
- Danihelka, I.; Wayne, G.; Uria, B.; Kalchbrenner, N.; Graves, A. Associative long short-term memory. In Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA, 20–22 June 2016; pp. 1986–1994. [Google Scholar]
- Hirose, A.; Yoshida, S. Generalization characteristics of complex-valued feedforward neural networks in relation to signal coherence. IEEE Trans. Neural Netw. Learn. Syst.
**2012**, 23, 541–551. [Google Scholar] [CrossRef] [PubMed] - Trabelsi, O.; Bilaniuk, Y.; Zhang, D.; Serdyuk, S.; Subramanian, J.; Santos, S.F.; Mehri, N.; Rostamzadeh, Y.; Bengio, C.; Pal, J. Deep Complex Networks. In Proceedings of the ICLR, Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Zhao, H.; Shi, J.; Qi, X.; Wang, X.; Jia, J. Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 2881–2890. [Google Scholar]
- Hou, Q.; Zhang, L.; Cheng, M.M.; Feng, J. Strip pooling: Rethinking spatial pooling for scene parsing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 4003–4012. [Google Scholar]
- Niu, Z.; Zhong, G.; Yu, H. A review on the attention mechanism of deep learning. Neurocomputing
**2021**, 452, 48–62. [Google Scholar] [CrossRef] - Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2980–2988. [Google Scholar]
- Rezaie, A.; Mauron, A.J.; Beyer, K. Sensitivity analysis of fractal dimensions of crack maps on concrete and masonry walls. Autom. Constr.
**2020**, 117, 103258. [Google Scholar] [CrossRef] - Li, L.; Sun, H.X.; Zhang, Y.; Yu, B. Surface cracking and fractal characteristics of bending fractured polypropylene fiber-reinforced geopolymer mortar. Fractal Fract.
**2021**, 5, 142. [Google Scholar] [CrossRef] - Wu, J.; Xie, D.; Yi, S.; Yin, S.; Hu, D.; Li, Y.; Wang, Y. Fractal Study of the Development Law of Mining Cracks. Fractal Fract.
**2023**, 7, 696. [Google Scholar] [CrossRef] - An, Q.; Chen, X.; Wang, H.; Yang, H.; Yang, Y.; Huang, W.; Wang, L. Segmentation of concrete cracks by using fractal dimension and UHK-net. Fractal Fract.
**2022**, 6, 95. [Google Scholar] [CrossRef] - Shi, Y.; Cui, L.; Qi, Z.; Meng, F.; Chen, Z. Automatic road crack detection using random structured forests. IEEE Trans. Intell. Transp. Syst.
**2016**, 17, 3434–3445. [Google Scholar] [CrossRef] - Yang, F.; Zhang, L.; Yu, S.; Prokhorov, D.; Mei, X.; Ling, H. Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans. Intell. Transp. Syst.
**2019**, 21, 1525–1535. [Google Scholar] [CrossRef] - Eisenbach, M.; Stricker, R.; Seichter, D.; Amende, K.; Debes, K.; Sesselmann, M.; Ebersbach, D.; Stoeckert, U.; Gross, H.M. How to get pavement distress detection ready for deep learning? A systematic approach. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 14–19 May 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 2039–2047. [Google Scholar]
- Yan, Y.; Zhu, S.; Ma, S.; Guo, Y.; Yu, Z. CycleADC-Net: A crack segmentation method based on multi-scale feature fusion. Measurement
**2022**, 204, 112107. [Google Scholar] [CrossRef] - Ali, R.; Chuah, J.H.; Talip, M.S.A.; Mokhtar, N.; Shoaib, M.A. Structural crack detection using deep convolutional neural networks. Autom. Constr.
**2022**, 133, 103989. [Google Scholar] [CrossRef] - Wang, C.Y.; Bochkovskiy, A.; Liao, H.Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023; pp. 7464–7475. [Google Scholar]
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. Ssd: Single shot multibox detector. In Proceedings of the Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Proceedings, Part I 14. Springer: Berlin/Heidelberg, Germany, 2016; pp. 21–37. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Proceedings, Part III 18. Springer: Berlin/Heidelberg, Germany, 2015; pp. 234–241. [Google Scholar]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818. [Google Scholar]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell.
**2017**, 39, 2481–2495. [Google Scholar] [CrossRef] [PubMed] - Zhou, X.P.; Huang, X.C.; Zhao, X.F. Optimization of the Critical Slip Surface of Three-Dimensional Slope by Using an Improved Genetic Algorithm. Int. J. Geomech.
**2020**, 20, 04020120. [Google Scholar] [CrossRef] - TruSoft International Inc. Benoit™; 1.01; TruSoft International Inc.: St. Petersburg, FL, USA, 1997; Available online: https://www.trusoft-international.com/ (accessed on 15 September 2023).

**Figure 12.**Fractal dimensions of different complex geometric patterns: (

**a**) line segments, (

**b**) squares, (

**c**) sixth-order Koch curves.

Method | MS (MB) | FPS (f/s) |
---|---|---|

Faster RCNN | 97.6 | 25.9 |

YOLO v5 | 58.3 | 30.1 |

YOLO v7 | 69.8 | 28.6 |

SSD | 65.9 | 57.8 |

Ours | 35.3 | 80.4 |

Method | Original | Low Brightness | ||||
---|---|---|---|---|---|---|

Pr | Re | F1 | Pr | Re | F1 | |

Faster RCNN | 85.80% | 87.10% | 86.45% | 82.20% | 83.60% | 82.89% |

YOLO v5s | 89.20% | 84.20% | 86.63% | 84.30% | 80.60% | 82.41% |

YOLO v7-tiny | 88.60% | 85.60% | 87.07% | 84.70% | 82.40% | 83.53% |

SSD | 86.40% | 85.90% | 86.15% | 79.80% | 73.40% | 76.47% |

Ours | 92.10% | 91.40% | 91.75% | 88.50% | 85.30% | 86.87% |

Method | Original | Low Brightness | ||||||
---|---|---|---|---|---|---|---|---|

Pr | Re | F1 | IoU | Pr | Recall | F1 | IoU | |

U-Net | 83.90% | 81.20% | 82.53% | 60.90% | 70.40% | 75.90% | 73.05% | 57.80% |

Deeplabv3+ | 84.70% | 85.00% | 84.85% | 63.70% | 72.80% | 74.60% | 73.69% | 53.40% |

PSPNet | 85.60% | 81.80% | 83.66% | 67.00% | 73.70% | 72.10% | 72.89% | 51.60% |

Seg-Net | 82.40% | 80.50% | 81.44% | 60.30% | 71.50% | 70.80% | 71.15% | 58.90% |

Ours | 86.30% | 89.20% | 87.73% | 68.00% | 76.30% | 78.10% | 77.19% | 62.60% |

Method | Line Segments | Squares | Sixth-Order Koch Curves |
---|---|---|---|

Benoit | 0.973 | 1.996 | 1.268 |

Ours | 0.986 | 1.994 | 1.267 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Cheng, J.; Chen, Q.; Huang, X.
An Algorithm for Crack Detection, Segmentation, and Fractal Dimension Estimation in Low-Light Environments by Fusing FFT and Convolutional Neural Network. *Fractal Fract.* **2023**, *7*, 820.
https://doi.org/10.3390/fractalfract7110820

**AMA Style**

Cheng J, Chen Q, Huang X.
An Algorithm for Crack Detection, Segmentation, and Fractal Dimension Estimation in Low-Light Environments by Fusing FFT and Convolutional Neural Network. *Fractal and Fractional*. 2023; 7(11):820.
https://doi.org/10.3390/fractalfract7110820

**Chicago/Turabian Style**

Cheng, Jiajie, Qiunan Chen, and Xiaocheng Huang.
2023. "An Algorithm for Crack Detection, Segmentation, and Fractal Dimension Estimation in Low-Light Environments by Fusing FFT and Convolutional Neural Network" *Fractal and Fractional* 7, no. 11: 820.
https://doi.org/10.3390/fractalfract7110820