Automatic Classification and Quantification of Asphalt Pavement Cracks Using Deep Learning-based Object Detection Algorithms

Document Type : Original Article

Authors
Hassan Hosseinzadeh 1
Ali Ghiami Bajgirani 2
Mohaddeseh Delavarian 3
1 M.Sc., Grad., Department of Civil Engineering, Technical and Engineering Faculty, Eqbal Lahori Institute of Higher Education, Mashhad, Iran.
2 Assistant Professor, Department of Civil Engineering, Technical and Engineering Faculty, Eqbal Lahori Institute of Higher Education, Mashhad, Iran.
3 Assistant Professor, Department of Computer Engineering, Technical and Engineering Faculty, Eqbal Lahori Institute of Higher Education, Mashhad, Iran.
10.22034/road.2022.350483.2066
Abstract
Pavement crack detection is an important method in road maintenance and traffic safety.Traditionally, road condition assessment was done by field inspection, which is time-consuming and costly, and the results are dependent on the evaluator's opinion, It should be replaced by automated methods to reduce both workload and thus maintenance costs. This paper examines the performance of YOLOv3 and YOLOv5 algorithms for automatic crack detection.These models are able to determine the type, position and geometric characteristics of the crack accurately and at a high speed compared to other methods.For the purpose of modeling, the images taken from Mashhad surface roads have been used. These images were labeled for linear and surface crack options. Then, models were created using the v3 model and five v5 series algorithms and transfer learning and were evaluated in terms of accuracy and prediction speed. The accuracy of the models is between 77 and 98% and the prediction speed of the models is between 17.4 and 105 milliseconds, which indicates the optimal performance of the models.Finally, v5s model was used as the final model for predicting cracks in one of Mashhad roads due to its acceptable accuracy (92.8) and high prediction speed (23.9 milliseconds) compared to other models.Then, based on the outputs of the model, the maintenance approach was presented.
Keywords
Pavement Management
Automatic Crack Detection
Deep Learning
YOLOv3
YOLOv5
-Bishop, C. M., & Nasrabadi, N. M. (2006). Pattern recognition and machine learning (Vol. 4). Springer.
-Bochkovskiy, A., Wang, C.-Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.
-Buduma, N., Buduma, N., & Papa, J. (2022). Fundamentals of deep learning. O'Reilly Media, Inc.
-Burkov, A. (2019). The hundred-page machine learning book, Vol. 1. Andriy Burkov Quebec City, QC, Canada.
-Chen, F.-C., & Jahanshahi, M. R. (2017). NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Transactions on Industrial Electronics, 65(5), 4392-4400.
-Cord, A., & Chambon, S. (2012). Automatic road defect detection by textural pattern recognition based on AdaBoost. Computer‐Aided Civil and Infrastructure Engineering, 27(4), 244-259.
-Desai, A. A. (2020). Asphalt pavement crack detection and classification using deep Convolutional Neural Networks Georgia Institute of Technology].
-Fei, Y., Wang, K. C., Zhang, A., Chen, C., Li, J. Q., Liu, Y., Yang, G., & Li, B. (2019). Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V. IEEE Transactions on Intelligent Transportation Systems, 21(1), 273-284.
-Gao, Y., & Mosalam, K. M. (2018). Deep transfer learning for image‐based structural damage recognition. Computer‐Aided Civil and Infrastructure Engineering, 33(9), 748-768.
-Girshick, R. (2015). Fast r-cnn. Proceedings of the IEEE international conference on computer vision,
-Gopalakrishnan, K., Khaitan, S. K., Choudhary, A., & Agrawal, A. (2017). Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Construction and building materials, 157, 322-330.
-Hoang, N.-D., & Nguyen, Q.-L. (2019). A novel method for asphalt pavement crack classification based on image processing and machine learning. Engineering with Computers, 35(2), 487-498.
-Hosseinzadeh, H., Ghiami Bajgirani, A., & Delavarian, M. (2022). Detection and classification of asphalt pavement cracks using YOLOv5. Journal of Modeling in Engineering.
-Hou, Y. (2019). Pixel-level Pavement Crack Detection Using Deep Convolutional Neural Network with Residual Blocks University of Missouri-Columbia].
-Hsieh, Y.-A., & Tsai, Y. J. (2020). Machine learning for crack detection: Review and model performance comparison. Journal of Computing in Civil Engineering, 34(5), 04020038.
-Hu, G. X., Hu, B. L., Yang, Z., Huang, L., & Li, P. (2021). Pavement Crack Detection Method Based on Deep Learning Models. Wireless Communications and Mobile Computing, 2021.
-Ian, G., Yoshua, B., & Aaron, C. (2016). Deep Learning (Adaptive Computation and Machine Learning Series). In: The MIT Press Cambridge, MA, USA.
-Jocher, G. (2020).  Retrieved 21 April from https://github.com/ultralytics/yolov5.
-Kelleher, J. D., Mac Namee, B., & D'arcy, A. (2020). Fundamentals of machine learning for predictive data analytics: algorithms, worked examples, and case studies. MIT PRESS.
-Kim, B., & Cho, S. (2018). Automated vision-based detection of cracks on concrete surfaces using a deep learning technique. Sensors, 18(10), 3452.
-Li, H., Zong, J., Nie, J., Wu, Z., & Han, H. (2021). Pavement crack detection algorithm based on densely connected and deeply supervised network. IEEE Access, 9, 11835-11842.
-Li, Q., Zou, Q., Zhang, D., & Mao, Q. (2011). FoSA: F* seed-growing approach for crack-line detection from pavement images. Image and Vision Computing, 29(12), 861-872.
-Li, S., Gu, X., Xu, X., Xu, D., Zhang, T., Liu, Z., & Dong, Q. (2021). Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Construction and Building Materials, 273, 121949.
-Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017). Feature pyramid networks for object detection. Proceedings of the IEEE conference on computer vision and pattern recognition,
-Liu, B., Zhao, W., & Sun, Q. (2017). Study of object detection based on Faster R-CNN. 2017 Chinese Automation Congress (CAC),
-Liu, C., Tao, Y., Liang, J., Li, K., & Chen, Y. (2018). Object detection based on YOLO network. 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC).
-Liu, F., Xu, G., Yang, Y., Niu, X., & Pan, Y. (2008). Novel approach to pavement cracking automatic detection based on segment extending. 2008 International Symposium on Knowledge Acquisition and Modeling.
-Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T., & Omata, H. (2018). Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454.
-Magazine, A. I. (2020). Guide to Yolov5 for Real-Time-Object Detection. Retrieved 10 February from https://analyticsindiamag.com/yolov5.
-Mandal, V., Uong, L., & Adu-Gyamfi, Y. (2018). Automated road crack detection using deep convolutional neural networks. 2018 IEEE International Conference on Big Data (Big Data).
-Nguyen, H.-N., Kam, T.-Y., & Cheng, P.-Y. (2014). An automatic approach for accurate edge detection of concrete crack utilizing 2D geometric features of crack. Journal of Signal Processing Systems, 77(3), 221-240.
-Nie, M., & Wang, C. (2019). Pavement Crack Detection based on yolo v3. 2019 2nd International Conference on Safety Produce Informatization (IICSPI).
-Pauly, L., Hogg, D., Fuentes, R., & Peel, H. (2017). Deeper networks for pavement crack detection. Proceedings of the 34th ISARC.
-Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
-Rosa, P. (2012). Automatic pavement crack detection and classification system. Transportation Research Board, National Research Council, Washington, DC, 57-65.
-Shahin, M. Y. (1994). Pavement management for airports, roads, and parking lots.
-Shi, Y., Cui, L., Qi, Z., Meng, F., & Chen, Z. (2016). Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 17(12), 3434-3445.
-Soviany, P., & Ionescu, R. T. (2018). Optimizing the trade-off between single-stage and two-stage deep object detectors using image difficulty prediction. 2018 20th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC).
-Thuan, D. (2021). Evolution of yolo algorithm and yolov5: the state-of-the-art object detection algorithm.
-Wang, X., & Hu, Z. (2017). Grid-based pavement crack analysis using deep learning. 2017 4th international conference on transportation information and safety (ICTIS).
-Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big data, 3(1), 1-40.
-Wu, W., Liu, H., Li, L., Long, Y., Wang, X., Wang, Z., Li, J., & Chang, Y. (2021). Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image. PloS one, 16(10), e0259283.
-Yang, C., Chen, J., Li, Z., & Huang, Y. (2021). Structural crack detection and recognition based on deep learning. Applied sciences, 11(6), 2868.
-Zhang, A., Wang, K. C., Fei, Y., Liu, Y., Tao, S., Chen, C., Li, J. Q., & Li, B. (2018). Deep
 learning–based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet. Journal of Computing in Civil Engineering, 32(5), 04018041.
-Zhang, A., Wang, K. C., Li, B., Yang, E., Dai, X., Peng, Y., Fei, Y., Liu, Y., Li, J. Q., & Chen, C. (2017). Automated pixel‐level pavement crack detection on 3D asphalt surfaces using a deep‐learning network. Computer‐Aided Civil and Infrastructure Engineering, 32(10), 805-819.
-Zhang, K., Cheng, H., & Zhang, B. (2018). Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning. Journal of Computing in Civil Engineering, 32(2), 04018001.
-Zhang, L., Yang, F., Zhang, Y. D., & Zhu, Y. J. (2016). Road crack detection using deep convolutional neural network. 2016 IEEE international conference on image processing (ICIP).
-Zou, Q., Zhang, Z., Li, Q., Qi, X., Wang, Q., & Wang, S. (2018). Deepcrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 28(3), 1498-1512.