| Research Article |
|
|
|
|
|
Semantic segmentation method of hydraulic structure crack based on feature enhancement |
| CHEN Bo1, ZHANG Hua1,2, CHEN Yongcan3,4, LI Yonglong5,6, XIONG Jinsong7 |
1. School of Information Engineering, Southwest University of Science and Technology, Mianyang 621000, China;
2. Innovation Research Institute of Sichuan Tianfu New District, Southwest University of Science and Technology, Chengdu 621010, China;
3. State Key Laboratory of Hydroscience and Engineering, Tsinghua University, Beijing 100084, China;
4. College of Civil Engineering and Surveying and Mapping, Southwest Petroleum University, Chengdu 610500, China;
5. Department of Electronic Engineering, Tsinghua University, Beijing 100084, China;
6. Sichuan Energy Internet Research Institute, Tsinghua University, Chengdu 610213, China;
7. Chongqing Hongyan Construction Machinery Manufacturing Co., Ltd., Chongqing 400712, China |
|
|
|
|
Abstract [Objective] Scientific, comprehensive, and standardized health monitoring is critical in the operation and maintenance of all types of water conservancy infrastructure. In this study, intelligent equipment is used to capture crack images of concrete dams and corridor hydraulic engineering scenes, and an artificial intelligence algorithm is used to achieve accurate recognition of crack information. However, most current research on concrete crack recognition lacks the analysis of crack information and simply obtains crack features through convolution and pooling to form a feature extraction network. The extracted high-dimensional features are not enhanced further, so the recognition effect cannot be continuously improved. A semantic segmentation technique for feature enhancement is proposed to solve the problem of low accuracy of crack location in the automatic detection of concrete cracks. [Methods] Statistical theory is used in this study to assess the pixel values of the cracked and non-cracked regions in three color channels and the proportion of the cracked region in the image. The size relationship and corresponding distribution of cracked and non-cracked regions on the pixel level are also obtained. Then, the ResNet-152 feature extraction network based on the residual structure is used to extract high-dimensional abstract semantic features from crack images. Due to the particularity of the residual structure, it can effectively reduce the loss of crack information during feature transmission and improve feature interoperability between different layers of the network so as to avoid the problem of gradient disappearance or explosion. Then, based on the results of statistical analysis, high-dimensional abstract features are sampled into two coarse segmentation feature maps corresponding to cracks and non-cracks. The similarity between the high-dimensional abstract features and the coarse segmentation feature map is calculated, the results of which are then used as weights to update high-dimensional abstract features to realize regional clustering of them. Finally, the clustered features are combined with the high-dimensional abstract features to obtain the enhanced features, which improve the crack location performance of the model. Meanwhile, the network loss function is optimized based on the crack information distribution. By controlling the number of samples used in the calculation of loss value, the contribution rate of crack information and non-crack information to the total loss value is balanced. As a result, the recognition accuracy of crack information is improved. [Results] We used an unmanned aerial vehicle and an orbital robot to capture images of two hydraulic engineering scenes, including the dam and the corridor. After image preprocessing and labeling, we obtained a total of 3 000 crack images and labels, including 1 000 dam crack images and 500 corridor crack images. We stratified the data set into a training set, a validation set, and a test set in an 8∶1∶1 ratio. The crack pixel accuracy, recall rate, intersection-over-unions, and overall total pixel accuracy of the model on the test set reached 92.48%, 86.52%, 80.82%, and 99.79%, respectively. [Conclusions] By analyzing the relationship and distribution of pixel values between crack information and non-crack information in crack images and using them as prior information to construct a feature enhancement network and design the objective function of network optimization, the shortcomings of current concrete crack identification methods can be effectively overcome, and the performance of the network to recognize crack information can be improved.
|
| Keywords
hydraulic cracks
convolutional neural network
statistical analysis
semantic segmentation
|
|
Issue Date: 27 June 2023
|
|
|
[1] YE X W, DONG C Z, LIU T. A review of machine vision-based structural health monitoring:Methodologies and applications[J]. Journal of Sensors, 2016, 2016:7103039.
[2] FENG D M, FENG M Q. Computer vision for SHM of civil infrastructure:From dynamic response measurement to damage detection-A review[J]. Engineering Structures, 2018, 156:105-117.
[3] XU Y, BROWNJOHN J M W. Review of machine-vision based methodologies for displacement measurement in civil structures[J]. Journal of Civil Structural Health Monitoring, 2018, 8(1):91-110.
[4] SPENCER B F JR, HOSKERE V, NARAZAKI Y. Advances in computer vision-based civil infrastructure inspection and monitoring[J]. Engineering, 2019, 5(2):199-222.
[5] ABDEL-QADER I, ABUDAYYEH O, KELLY M E. Analysis of edge-detection techniques for crack identification in bridges[J]. Journal of Computing in Civil Engineering, 2003, 17(4):255-263.
[6] LI G, ZHAO X X, DU K, et al. Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine[J]. Automation in Construction, 2017, 78:51-61.
[7] YU S N, JANG J H, HAN C S. Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel[J]. Automation in Construction, 2007, 16(3):255-261.
[8] LI G, HE S H, JU Y F, et al. Long-distance precision inspection method for bridge cracks with image processing[J]. Automation in Construction, 2014, 41:83-95.
[9] YAMAGUCHI T, HASHIMOTO S. Fast crack detection method for large-size concrete surface images using percolation-based image processing[J]. Machine Vision and Applications, 2010, 21(5):797-809.
[10] FUJITA Y, HAMAMOTO Y. A robust method for automatically detecting cracks on noisy concrete surfaces[C]//22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Tainan, China:Springer, 2009:76-85.
[11] LIU Z W, SUANDI S A, OHASHI T, et al. Tunnel crack detection and classification system based on image processing[C]//Proceedings Volume 4664, Machine Vision Applications in Industrial Inspection X. San Jose, USA:SPIE, 2002:145-152.
[12] GAVILÁN M, BALCONES D, MARCOS O, et al. Adaptive road crack detection system by pavement classification[J]. Sensors, 2011, 11(10):9628-9657.
[13] SHI Y, CUI L M, QI Z Q, et al. Automatic road crack detection using random structured forests[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12):3434-3445.
[14] CHA Y J, CHOI W, BVYVKÖZTVRK O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5):361-378.
[15] ZHANG L, YANG F, ZHANG Y D, et al. Road crack detection using deep convolutional neural network[C]//2016 IEEE International Conference on Image Processing. Phoenix:IEEE, 2016:3708-3712.
[16] 陈波,张华,王姮,等.基于迁移学习的坝面表观缺陷智能检测方法研究[J].水利水电技术, 2020, 51(4):106-112. CHEN B, ZHANG H, WANG H, et al. Transfer learning-based study on method of intelligent detection of dam surface apparent defect[J]. Water Resources and Hydropower Engineering, 2020, 51(4):106-112.(in Chinese)
[17] ZHANG Y X, HUANG J, CAI F H. On bridge surface crack detection based on an improved YOLO v3 algorithm[J]. IFAC-PapersOnLine, 2020, 53(2):8205-8210.
[18] YU Z W, SHEN Y G, SHEN C K. A real-time detection approach for bridge cracks based on YOLOv4-FPM[J]. Automation in Construction, 2021, 122:103514.
[19] 李太文,范昕炜.基于Faster R-CNN的道路裂缝识别[J].电子技术应用, 2020, 46(7):53-56, 59. LI T W, FAN X W. Road crevice recognition based on Faster R-CNN[J]. Application of Electronic Technique, 2020, 46(7):53-56, 59.(in Chinese)
[20] DUNG C V, ANH L D. Autonomous concrete crack detection using deep fully convolutional neural network[J]. Automation in Construction, 2019, 99:52-58.
[21] LI S Y, ZHAO X F, ZHOU G Y. Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(7):616-634.
[22] FENG C C, ZHANG H, WANG H R, et al. Automatic pixel-level crack detection on dam surface using deep convolutional network[J]. Sensors, 2020, 20(7):2069.
[23] CHEN J, HE Y. A novel U-shaped encoder-decoder network with attention mechanism for detection and evaluation of road cracks at pixel level[J]. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(13):1721-1736.
[24] YANG F, ZHANG L, YU S J, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4):1525-1535.
[25] CHEN B, ZHANG H, LI Y L, et al. Quantify pixel-level detection of dam surface crack using deep learning[J]. Measurement Science and Technology, 2022, 33(6):065402.
[26] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA:Curran Associates Inc., 2017:6000-6010.
[27] LIU Y H, YAO J, LU X H, et al. DeepCrack:A deep hierarchical feature learning architecture for crack segmentation[J]. Neurocomputing, 2019, 338:139-153.
[28] BANG S, PARK S, KIM H, et al. Encoder-decoder network for pixel-level road crack detection in black-box images[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8):713-727. |
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
2023,
Vol. 63
