Detection algorithms and optimization of image processing for visual sensors using combined laser structured light
SUN Bowen, ZHU Zhiming, GUO Jichang, ZHANG Tianyi
Key Laboratory for Advanced Materials Processing Technology of Ministry of Education, Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China
摘要基于激光结构光的视觉传感器广泛应用于焊接领域的坡口检测和焊缝跟踪。该文提出了一种基于组合激光结构光的新型视觉传感器,独特的光路结构设计避免了传感器应用于不同焊接位姿时繁琐的外参数标定,仅依靠传感器内部固有参数(应用前需校准)和焊接坡口图像的特征点坐标值,即可实现焊接坡口参数的在线检测,有效增强了传感器的适应性。通过对不同图像处理算法的改进和合理组合,对图像处理流程进行了优化。动态感兴趣区(region of interest,ROI)区域提取算法可快速寻获有价值的激光线和特征点所在区域,有效提升了后续图像处理速度;顶帽变换与自适应二值化组合,在将激光线灰度值均匀化的同时,实现了激光线与背景图像的有效区分。运用基于LOG(Laplacian of Gaussian)算子的边缘识别算法,可提取出激光线的单像素边缘;采用最小二乘法对所求得的激光中心线离散点进行直线拟合,通过联立直线方程求交点的方式,实现了对焊接坡口特征点图像坐标值的准确识别。借助Visual Studio平台,运用改进的图像处理算法、优化的图像处理流程和检测算法,对特征参数不同的V形焊接坡口进行检测试验,检测误差均在±4%以内,验证了所提出视觉传感器及其检测算法和图像处理流程的合理性和适用性。
Abstract:Visual sensors based on laser structured light are widely used in the welding field for detection of welding grooves and tracking of weld seams. This paper describes a visual sensor based on combined laser structured light where the light paths within the visual sensor are designed to eliminate the need to calibrate the external sensor parameters for different positions and postures to provide on-line detection of welding groove parameters using only the internal sensor parameters (which require calibration) and the feature point coordinates of the welding groove image. Thus, this method is much more adaptable than previous methods. The image processing is optimized by improving and combining various image processing algorithms. A dynamic region of interest region extraction algorithm is used to quickly find the region that includes the laser lines and feature points to significantly improve the subsequent image processing speed. Then, a top hat transformation and self-adaptive binarization are used to homogenize the gray values of the laser lines that differentiates the laser lines from the background. After that, an edge recognition algorithm based on the Laplacian of Gaussian (LOG) operator is used to extract the single pixel edges of the laser lines which are used to then identify the discrete points of the laser center lines that are fit to linear equations by the least squares method. Finally, these linear equations are used to accurately calculate the image coordinates of the feature points of the welding groove. All these image processing algorithms were implemented in Visual Studio for tests to detect V-shaped welding grooves with various characteristic parameters. The detection errors are all within ±4%, which shows that the detection algorithms are reasonable and applicable for image processing of welding processes with visual sensors.
[1] 罗璐, 林三宝, 伏喜斌, 等. 基于激光视觉的焊缝特征信息检测技术研究[J]. 焊接, 2008(7):33-37. LUO L, LIN S B, FU X B, et al. Inspection technology on joint feature information based on laser vision[J]. Welding & Joining, 2008(7):33-37. (in Chinese) [2] LEE S K, NA S J. A study on automatic seam tracking in pulsed laser edge welding by using a vision sensor without an auxiliary light source[J]. Journal of Manufacturing Systems, 2002, 21(4):302-315. [3] 于英飞, 郭吉昌, 朱志明. 焊接可视化技术的发展与研究现状[J]. 焊接, 2017(12):4-8. YU Y F, GUO J C, ZHU Z M. Development and research status of welding visualization technology[J]. Welding & Joining, 2017(12):4-8. (in Chinese) [4] 郭吉昌, 朱志明, 于英飞, 等. 焊接领域激光结构光视觉传感技术的研究及应用[J]. 中国激光, 2017, 44(12):7-16. GUO J C, ZHU Z M, YU Y F, et al. Research and application of visual sensing technology based on laser structured light in welding industry[J]. Chinese Journal of Laser, 2017, 44(12):7-16. (in Chinese) [5] 章妍. 基于铝合金GTAW焊三光路视觉传感的熔池特征提取[D]. 上海:上海交通大学, 2014. ZHANG Y. Visual characters extraction of weld pool based on three path meseurement during aluminum alloy GTAW process[D]. Shanghai:Shanghai Jiao Tong University, 2014. (in Chinese) [6] 刘志辉. 坡口及焊缝数字轮廓的激光光栅视觉测量[D]. 兰州:兰州理工大学, 2018. LIU Z H. Laser grating vision measurement of groove and weld digital contour[D]. Lanzhou:Lanzhou University of Technology, 2018. (in Chinese) [7] 冈萨雷斯, 伍兹. 数字图像处理[M]. 阮秋琦, 阮宇智, 译. 北京:电子工业出版社. 2011. GONZALEZ R C, WOODS R E. Digital image processing[M]. RUAN Q Q, RUAN Y Z, trans. Beijing:Publishing House of Electronic Industry. 2011. (in Chinese) [8] 刘子腾, 白瑞林, 王秀平. 基于激光视觉的角焊缝图像特征点提取[J]. 焊接学报, 2016, 37(2):89-93. LIU Z T, BAI R L, WANG X P. Image feature extraction of fillet weld based on laser vision[J]. Transactions of the China Welding Institution, 2016, 37(2):89-93. (in Chinese) [9] 郭吉昌, 朱志明, 孙博文. 基于视觉的焊枪空间位姿实时检测技术[J]. 电焊机, 2018, 48(9):1-7. GUO J C, ZHU Z M, SUN B W. Real-time detection of position and attitude for welding torch based on vision[J]. Electric Welding Machine, 2018, 48(9):1-7. (in Chinese) [10] GUO J C, ZHU Z M, SUN B W, et al. Principle of an innovative visual sensor based on combined laser structured lights and its experimental verification[J]. Optics & Laser Technology, 2019, 111:35-44. [11] GUO J C, ZHU Z M, SUN B W, et al. A novel multifunctional visual sensor based on combined laser structured lights and its anti-jamming detection algorithms[J]. Welding in the World, 2018. DOI:10.1007/s40194-018-0654-y. [12] 王璇. 焊接机器人焊缝信息视觉检测系统研究[D]. 天津:河北工业大学, 2013. WANG X. Research on visual detection sensor of welding robot[D]. Tianjin:Hebei University of Technology. (in Chinese) [13] 赵军. 大构件焊缝磨抛机器人视觉测量技术的研究[D]. 长春:吉林大学, 2014. ZHAO J. Research on grinding/polishing robotic vision measurement technology for weld beads of large structural parts[D]. Changchun:Jilin University, 2014. (in Chinese) [14] 沈同平, 王元茂, 方芳, 等. 基于灰度形态学的图像边缘特征提取算法研究[J]. 兰州文理学院学报(自然科学版), 2018, 32(2):58-61. SHEN T P, WANG Y M, FANG F, et al. Edge feature extraction algorithm based on gray morphology[J]. Journal of Lanzhou University of Arts and Science (Natural Sciences), 2018, 32(2):58-61. (in Chinese) [15] 范九伦, 赵凤. 灰度图像的二维Otsu曲线阈值分割法[J]. 电子学报, 2007, 35(4):751-755. FAN J L, ZHAO F. Two-dimensional Otsu's curve thresholding segmentation method for gray-level images[J]. Acta Electronica Sinica, 2007, 35(4):751-755. (in Chinese) [16] 郭佳. 基于光照不均匀图像的自适应二值化方法研究[D]. 武汉:武汉科技大学, 2013. GUO J. The research of binarization method based on non-uniform illumination images[D]. Wuhan:Wuhan University of Science and Technology, 2013. (in Chinese) [17] 陈铖. 焊缝跟踪系统中基于C++的图像处理技术研究[D]. 成都:西南交通大学, 2017. CHEN C. Research on image processing technology based on C++ in weld tracking system[D]. Chengdu:Southwest Jiaotong University, 2017. (in Chinese) [18] 刘习文, 王国荣, 石永华. 单条纹激光引导焊缝跟踪图像处理[J]. 焊接学报, 2006, 27(6):25-28, 32. LIU X W, WANG G R, SHI Y H. Image processing in welding seam tracking based on single stripe laser[J]. Transactions of the China Welding Institution, 2006, 27(6):25-28, 32. (in Chinese) [19] 刘蕊. 数字图像中边缘检测算法的研究[D]. 镇江:江苏科技大学, 2009. LIU R. Digital image edge detection algorithm[D]. Zhenjiang:Jiangsu University of Science and Technology, 2009. (in Chinese) [20] 张铮, 徐超, 任淑霞, 等. 数字图像处理与机器视觉[M]. 2版. 北京:人民邮电出版社, 2014. ZHANG Z, XU C, REN S X, et al. Digital image processing and machine vision[M]. 2nd ed. Beijing:Post & Telecom Press, 2014. (in Chinese) [21] 管力明, 李磊, 林剑. 基于改进LOG算子的图像边缘检测方法[J]. 机电工程, 2010, 27(12):113-115. GUAN L M, LI L, LIN J. Image edge detection method based on improved LOG operator[J]. Journal of Mechanical & Electrical Engineering, 2010, 27(12):113-115. (in Chinese) [22] 李莹莹, 张志毅, 袁林. 线结构光光条中心提取综述[J]. 激光与光电子学进展, 2013, 50(10):100002. LI YY, ZHANG Z Y, YUAN L. Survey on linear structured light stripe center extraction[J]. Laser & Optoelectronics Progress, 2013, 50(10):100002. (in Chinese)