Please wait a minute...
 首页  期刊介绍 期刊订阅 联系我们 横山亮次奖 百年刊庆
 
最新录用  |  预出版  |  当期目录  |  过刊浏览  |  阅读排行  |  下载排行  |  引用排行  |  横山亮次奖  |  百年刊庆
清华大学学报(自然科学版)  2023, Vol. 63 Issue (7): 1135-1143    DOI: 10.16511/j.cnki.qhdxxb.2023.26.009
  论文 本期目录 | 过刊浏览 | 高级检索 |
基于特征增强的水工结构裂缝语义分割方法
陈波1, 张华1,2, 陈永灿3,4, 李永龙5,6, 熊劲松7
1. 西南科技大学 信息工程学院, 绵阳 621000;
2. 西南科技大学 西南科大四川天府新区创新研究院, 成都 621010;
3. 清华大学 水沙科学与水利水电工程国家重点实验室, 北京 100084;
4. 西南石油大学 土木工程与测绘学院, 成都 610500;
5. 清华大学 电子工程系, 北京 100084;
6. 清华四川能源互联网研究院, 成都 610213;
7. 重庆红岩建设机械制造有限责任公司, 重庆 400712
Semantic segmentation method of hydraulic structure crack based on feature enhancement
CHEN Bo1, ZHANG Hua1,2, CHEN Yongcan3,4, LI Yonglong5,6, XIONG Jinsong7
1. School of Information Engineering, Southwest University of Science and Technology, Mianyang 621000, China;
2. Innovation Research Institute of Sichuan Tianfu New District, Southwest University of Science and Technology, Chengdu 621010, China;
3. State Key Laboratory of Hydroscience and Engineering, Tsinghua University, Beijing 100084, China;
4. College of Civil Engineering and Surveying and Mapping, Southwest Petroleum University, Chengdu 610500, China;
5. Department of Electronic Engineering, Tsinghua University, Beijing 100084, China;
6. Sichuan Energy Internet Research Institute, Tsinghua University, Chengdu 610213, China;
7. Chongqing Hongyan Construction Machinery Manufacturing Co., Ltd., Chongqing 400712, China
全文: PDF(8050 KB)   HTML
输出: BibTeX | EndNote (RIS)      
摘要 基于计算机视觉的混凝土裂缝自动检测方法逐渐成为大坝、廊道和引水隧洞等水工结构场景检测任务的主流选择。然而,目前大多数方法在裂缝特征提取过程中均存在不同程度的损耗,缺乏针对性的补偿措施,导致最终检测效果不佳。该文提出了一种基于特征增强的水工结构裂缝语义分割方法,主要用于解决混凝土水工结构裂缝高精度语义分割问题。该方法通过对裂缝数据进行统计学分析,获取裂缝像素与非裂缝像素关系及其对应分布情况;采用ResNet-152特征提取网络提取裂缝图像抽象语义信息,并根据统计分析结果对高维特征进行区域聚集,构建自注意力模块,增强模型对裂缝的定位性能;结合裂缝信息分布情况,对网络损失函数进行优化,增加裂缝特征对总体损失值的贡献率,提升模型对裂缝的识别精度。该文采用智能化设备获取大坝和廊道2种水工结构场景的图像数据,图像数据经图像预处理和标注整理后获得的裂缝图像和标签共3 000张;将由训练获得的分割模型在测试集上进行测试,裂缝像素准确率、召回率、交并比和总像素准确率分别达92.48%、86.52%、80.82%和99.79%。该文提出的分割方法在水工结构裂缝检测方面具有一定应用研究价值和推广意义。
服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
陈波
张华
陈永灿
李永龙
熊劲松
关键词 水工裂缝卷积神经网络统计学分析语义分割    
Abstract:[Objective] Scientific, comprehensive, and standardized health monitoring is critical in the operation and maintenance of all types of water conservancy infrastructure. In this study, intelligent equipment is used to capture crack images of concrete dams and corridor hydraulic engineering scenes, and an artificial intelligence algorithm is used to achieve accurate recognition of crack information. However, most current research on concrete crack recognition lacks the analysis of crack information and simply obtains crack features through convolution and pooling to form a feature extraction network. The extracted high-dimensional features are not enhanced further, so the recognition effect cannot be continuously improved. A semantic segmentation technique for feature enhancement is proposed to solve the problem of low accuracy of crack location in the automatic detection of concrete cracks. [Methods] Statistical theory is used in this study to assess the pixel values of the cracked and non-cracked regions in three color channels and the proportion of the cracked region in the image. The size relationship and corresponding distribution of cracked and non-cracked regions on the pixel level are also obtained. Then, the ResNet-152 feature extraction network based on the residual structure is used to extract high-dimensional abstract semantic features from crack images. Due to the particularity of the residual structure, it can effectively reduce the loss of crack information during feature transmission and improve feature interoperability between different layers of the network so as to avoid the problem of gradient disappearance or explosion. Then, based on the results of statistical analysis, high-dimensional abstract features are sampled into two coarse segmentation feature maps corresponding to cracks and non-cracks. The similarity between the high-dimensional abstract features and the coarse segmentation feature map is calculated, the results of which are then used as weights to update high-dimensional abstract features to realize regional clustering of them. Finally, the clustered features are combined with the high-dimensional abstract features to obtain the enhanced features, which improve the crack location performance of the model. Meanwhile, the network loss function is optimized based on the crack information distribution. By controlling the number of samples used in the calculation of loss value, the contribution rate of crack information and non-crack information to the total loss value is balanced. As a result, the recognition accuracy of crack information is improved. [Results] We used an unmanned aerial vehicle and an orbital robot to capture images of two hydraulic engineering scenes, including the dam and the corridor. After image preprocessing and labeling, we obtained a total of 3 000 crack images and labels, including 1 000 dam crack images and 500 corridor crack images. We stratified the data set into a training set, a validation set, and a test set in an 8∶1∶1 ratio. The crack pixel accuracy, recall rate, intersection-over-unions, and overall total pixel accuracy of the model on the test set reached 92.48%, 86.52%, 80.82%, and 99.79%, respectively. [Conclusions] By analyzing the relationship and distribution of pixel values between crack information and non-crack information in crack images and using them as prior information to construct a feature enhancement network and design the objective function of network optimization, the shortcomings of current concrete crack identification methods can be effectively overcome, and the performance of the network to recognize crack information can be improved.
Key wordshydraulic cracks    convolutional neural network    statistical analysis    semantic segmentation
收稿日期: 2022-10-27      出版日期: 2023-06-27
基金资助:国家自然科学基金资助项目(U21A20157);四川省科技计划项目(2022YFSY0011,2022YFQ0080,2023YFS0410)
通讯作者: 李永龙,高级工程师,E-mail:liyonglong@hotmail.com     E-mail: liyonglong@hotmail.com
作者简介: 陈波(1994—),男,博士研究生。
引用本文:   
陈波, 张华, 陈永灿, 李永龙, 熊劲松. 基于特征增强的水工结构裂缝语义分割方法[J]. 清华大学学报(自然科学版), 2023, 63(7): 1135-1143.
CHEN Bo, ZHANG Hua, CHEN Yongcan, LI Yonglong, XIONG Jinsong. Semantic segmentation method of hydraulic structure crack based on feature enhancement. Journal of Tsinghua University(Science and Technology), 2023, 63(7): 1135-1143.
链接本文:  
http://jst.tsinghuajournals.com/CN/10.16511/j.cnki.qhdxxb.2023.26.009  或          http://jst.tsinghuajournals.com/CN/Y2023/V63/I7/1135
  
  
  
  
  
  
  
  
  
  
[1] YE X W, DONG C Z, LIU T. A review of machine vision-based structural health monitoring:Methodologies and applications[J]. Journal of Sensors, 2016, 2016:7103039.
[2] FENG D M, FENG M Q. Computer vision for SHM of civil infrastructure:From dynamic response measurement to damage detection-A review[J]. Engineering Structures, 2018, 156:105-117.
[3] XU Y, BROWNJOHN J M W. Review of machine-vision based methodologies for displacement measurement in civil structures[J]. Journal of Civil Structural Health Monitoring, 2018, 8(1):91-110.
[4] SPENCER B F JR, HOSKERE V, NARAZAKI Y. Advances in computer vision-based civil infrastructure inspection and monitoring[J]. Engineering, 2019, 5(2):199-222.
[5] ABDEL-QADER I, ABUDAYYEH O, KELLY M E. Analysis of edge-detection techniques for crack identification in bridges[J]. Journal of Computing in Civil Engineering, 2003, 17(4):255-263.
[6] LI G, ZHAO X X, DU K, et al. Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine[J]. Automation in Construction, 2017, 78:51-61.
[7] YU S N, JANG J H, HAN C S. Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel[J]. Automation in Construction, 2007, 16(3):255-261.
[8] LI G, HE S H, JU Y F, et al. Long-distance precision inspection method for bridge cracks with image processing[J]. Automation in Construction, 2014, 41:83-95.
[9] YAMAGUCHI T, HASHIMOTO S. Fast crack detection method for large-size concrete surface images using percolation-based image processing[J]. Machine Vision and Applications, 2010, 21(5):797-809.
[10] FUJITA Y, HAMAMOTO Y. A robust method for automatically detecting cracks on noisy concrete surfaces[C]//22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Tainan, China:Springer, 2009:76-85.
[11] LIU Z W, SUANDI S A, OHASHI T, et al. Tunnel crack detection and classification system based on image processing[C]//Proceedings Volume 4664, Machine Vision Applications in Industrial Inspection X. San Jose, USA:SPIE, 2002:145-152.
[12] GAVILÁN M, BALCONES D, MARCOS O, et al. Adaptive road crack detection system by pavement classification[J]. Sensors, 2011, 11(10):9628-9657.
[13] SHI Y, CUI L M, QI Z Q, et al. Automatic road crack detection using random structured forests[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12):3434-3445.
[14] CHA Y J, CHOI W, BVYVKÖZTVRK O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5):361-378.
[15] ZHANG L, YANG F, ZHANG Y D, et al. Road crack detection using deep convolutional neural network[C]//2016 IEEE International Conference on Image Processing. Phoenix:IEEE, 2016:3708-3712.
[16] 陈波,张华,王姮,等.基于迁移学习的坝面表观缺陷智能检测方法研究[J].水利水电技术, 2020, 51(4):106-112. CHEN B, ZHANG H, WANG H, et al. Transfer learning-based study on method of intelligent detection of dam surface apparent defect[J]. Water Resources and Hydropower Engineering, 2020, 51(4):106-112.(in Chinese)
[17] ZHANG Y X, HUANG J, CAI F H. On bridge surface crack detection based on an improved YOLO v3 algorithm[J]. IFAC-PapersOnLine, 2020, 53(2):8205-8210.
[18] YU Z W, SHEN Y G, SHEN C K. A real-time detection approach for bridge cracks based on YOLOv4-FPM[J]. Automation in Construction, 2021, 122:103514.
[19] 李太文,范昕炜.基于Faster R-CNN的道路裂缝识别[J].电子技术应用, 2020, 46(7):53-56, 59. LI T W, FAN X W. Road crevice recognition based on Faster R-CNN[J]. Application of Electronic Technique, 2020, 46(7):53-56, 59.(in Chinese)
[20] DUNG C V, ANH L D. Autonomous concrete crack detection using deep fully convolutional neural network[J]. Automation in Construction, 2019, 99:52-58.
[21] LI S Y, ZHAO X F, ZHOU G Y. Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(7):616-634.
[22] FENG C C, ZHANG H, WANG H R, et al. Automatic pixel-level crack detection on dam surface using deep convolutional network[J]. Sensors, 2020, 20(7):2069.
[23] CHEN J, HE Y. A novel U-shaped encoder-decoder network with attention mechanism for detection and evaluation of road cracks at pixel level[J]. Computer-Aided Civil and Infrastructure Engineering, 2022, 37(13):1721-1736.
[24] YANG F, ZHANG L, YU S J, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4):1525-1535.
[25] CHEN B, ZHANG H, LI Y L, et al. Quantify pixel-level detection of dam surface crack using deep learning[J]. Measurement Science and Technology, 2022, 33(6):065402.
[26] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA:Curran Associates Inc., 2017:6000-6010.
[27] LIU Y H, YAO J, LU X H, et al. DeepCrack:A deep hierarchical feature learning architecture for crack segmentation[J]. Neurocomputing, 2019, 338:139-153.
[28] BANG S, PARK S, KIM H, et al. Encoder-decoder network for pixel-level road crack detection in black-box images[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8):713-727.
[1] 杨波, 邱雷, 吴书. 异质图神经网络协同过滤模型[J]. 清华大学学报(自然科学版), 2023, 63(9): 1339-1349.
[2] 杜晓闯, 梁漫春, 黎岢, 俞彦成, 刘欣, 汪向伟, 王汝栋, 张国杰, 付起. 基于卷积神经网络的γ放射性核素识别方法[J]. 清华大学学报(自然科学版), 2023, 63(6): 980-986.
[3] 邓青, 张博, 李宜豪, 周亮, 周正青, 蒋慧灵, 高扬. 基于级联CNN的疏散场景中人群数量估计模型[J]. 清华大学学报(自然科学版), 2023, 63(1): 146-152.
[4] 王晓萌, 管志斌, 辛伟, 王嘉捷. 基于深度卷积神经网络的源代码缺陷检测方法[J]. 清华大学学报(自然科学版), 2021, 61(11): 1267-1272.
[5] 韩坤, 潘海为, 张伟, 边晓菲, 陈春伶, 何舒宁. 基于多模态医学图像的Alzheimer病分类方法[J]. 清华大学学报(自然科学版), 2020, 60(8): 664-671,682.
[6] 孙博文, 张鹏, 成茗宇, 李新童, 李祺. 基于代码图像增强的恶意代码检测方法[J]. 清华大学学报(自然科学版), 2020, 60(5): 386-392.
[7] 林鹏, 魏鹏程, 樊启祥, 陈闻起. 基于CNN模型的施工现场典型安全隐患数据学习[J]. 清华大学学报(自然科学版), 2019, 59(8): 628-634.
[8] 梁杰, 陈嘉豪, 张雪芹, 周悦, 林家骏. 基于独热编码和卷积神经网络的异常检测[J]. 清华大学学报(自然科学版), 2019, 59(7): 523-529.
[9] 张思聪, 谢晓尧, 徐洋. 基于dCNN的入侵检测方法[J]. 清华大学学报(自然科学版), 2019, 59(1): 44-52.
[10] 刘琼, 李宗贤, 孙富春, 田永鸿, 曾炜. 基于深度信念卷积神经网络的图像识别与分类[J]. 清华大学学报(自然科学版), 2018, 58(9): 781-787.
[11] 芦效峰, 张胜飞, 伊胜伟. 基于CNN和RNN的自由文本击键模式持续身份认证[J]. 清华大学学报(自然科学版), 2018, 58(12): 1072-1078.
[12] 李晓飞, 许庆, 熊辉, 王建强, 李克强. 基于候选区域选择及深度网络模型的骑车人识别[J]. 清华大学学报(自然科学版), 2017, 57(5): 491-496.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
版权所有 © 《清华大学学报(自然科学版)》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn