3D reconstruction and inflection point identification of weld seams in complex environments

Xiaob ing FENG, Yixiang DAI, Tengyue HAN, Fei YUAN, Guijin WANG

Journal of Tsinghua University(Science and Technology) ›› 2025, Vol. 65 ›› Issue (10) : 1980-1991.

PDF(5795 KB)
PDF(5795 KB)
Journal of Tsinghua University(Science and Technology) ›› 2025, Vol. 65 ›› Issue (10) : 1980-1991. DOI: 10.16511/j.cnki.qhdxxb.2025.22.030
Electronic Engineering

3D reconstruction and inflection point identification of weld seams in complex environments

Author information +
History +

Abstract

Objective: Welding remains a key process in modern manufacturing, but it still faces considerable automation challenges due to the complex and noisy welding environments. Accurate weld seam geometry detection is critical for robotic seam tracking and welding quality assurance. In particular, detecting inflection points on the weld seam—typically corresponding to groove edges—is crucial for trajectory planning and control of welding robots. However, intense arc light, dynamic spatter, and reflective interference often contribute to image quality degradation, complicating the robust extraction of seam features. To address these challenges, this paper proposes a comprehensive framework for the three-dimensional (3D) reconstruction and inflection point detection of weld seams in complex environments, seeking to enhance noise robustness, spatial accuracy, and real-time performance in intelligent welding systems. Methods: The proposed method comprises the following three sequential modules: temporal denoising, laser stripe segmentation, and 3D inflection point detection. First, a Gaussian background modeling-based temporal filtering algorithm is developed to capture frame-wise variations and suppress transient noise, such as welding spatter. This algorithm adaptively classifies pixels as foreground or background using statistical thresholds, thereby enhancing the signal-to-noise ratio. Second, a lightweight deep segmentation model incorporating uncertainty modeling and nonlocal self-attention is introduced. This model employs a dual-stage architecture, i.e., an initial U-Net with attention to coarse segmentation, followed by a CriticNet-enhanced refinement stage guided by epistemic uncertainty maps. This strategy ensures continuity in stripe detection and robustness against weak exposure or partial occlusions. Finally, the segmented laser stripe centerlines are projected into 3D space based on calibrated structured-light principles. An optimal view normalization step ensures that the viewpoint is aligned vertically with the laser plane, and a hierarchical geometric model of the weld cross-section is created to assist in the localization of inflection points. To improve temporal consistency, a sliding-window filter smooths the extracted inflection trajectories, and a feedback loop updates the 3D model in real time, enhancing prediction stability. Results: The framework is tested on a custom dataset collected from real-world welding scenarios using a mobile welding robot equipped with a laser vision sensor. This dataset comprises over 21 000 frames captured under the following three representative conditions: nonuniform surfaces, intense reflection, and severe spatter. Quantitative evaluations demonstrate that the proposed method achieves a weld inflection point detection success rate of 78%, which is a 13.3% improvement over baseline methods. The 3D reconstruction accuracy remains within a submillimeter error margin (≤0.1 mm), with a maximum relative deviation of only 0.2%. Ablation studies indicate that each module—temporal filtering, deep segmentation, and 3D modeling—offers substantial contributions to overall performance improvement. Additionally, the segmentation model achieves a mean intersection over union (mIoU) of 83.5% and a recall rate of 88.1%, with only 0.67 million parameters, outperforming conventional U-Net and SegFormer baseline methods in accuracy and efficiency. Conclusions: This study introduces an effective and lightweight method to addressing the challenges of weld seam 3D reconstruction and inflection point detection in noisy environments. By combining temporal filtering, uncertainty-aware segmentation, and geometry-guided 3D analysis, the method demonstrates strong noise resilience and geometric accuracy. These results highlight its strong potential for real-time application in intelligent welding systems, supporting accurate seam tracking and robotic welding guidance in complex industrial environments.

Key words

intelligent welding / 3D reconstruction / laser vision / deep learning / inflection point detection

Cite this article

Download Citations
Xiaob ing FENG , Yixiang DAI , Tengyue HAN , et al . 3D reconstruction and inflection point identification of weld seams in complex environments[J]. Journal of Tsinghua University(Science and Technology). 2025, 65(10): 1980-1991 https://doi.org/10.16511/j.cnki.qhdxxb.2025.22.030

References

1
郭岩宝, 王斌, 王德国, 等. 焊接机器人的研究进展与发展趋势[J]. 现代制造工程, 2021 (5): 53- 63.
GUO Y B , WANG B , WANG D G , et al. Research progress and development trend of welding robot[J]. Modern Manufacturing Engineering, 2021 (5): 53- 63.
2
王斌, 温磊. 我国新型钢铁材料及焊接性与焊接材料的发展[J]. 环球市场, 2016 (24): 178.
WANG B , WEN L . The development of new steel materials and their weldability and welding materials in China[J]. Global Market, 2016 (24): 178.
3
丁少闻, 张小虎, 于起峰, 等. 非接触式三维重建测量方法综述[J]. 激光与光电子学进展, 2017, 54 (7): 070003.
DING S W , ZHANG X H , YU Q F , et al. Overview of non-contact 3D reconstruction measurement methods[J]. Laser & Optoelectronics Progress, 2017, 54 (7): 070003.
4
张弓, 脱帅华, 曹学鹏, 等. 焊接机器人焊缝跟踪技术的现状与发展趋势[J]. 科学技术与工程, 2021, 21 (10): 3868- 3876.
ZHANG G , TUO S H , CAO X P , et al. The state-of-the-art and developing trends of weld seam tracking technology of welding robot[J]. Science Technology and Engineering, 2021, 21 (10): 3868- 3876.
5
刘康, 刘翠荣, 吴志生, 等. 主动视觉的焊缝跟踪图像分析[J]. 焊接技术, 2020, 49 (2): 75-78, 6.
LIU K , LIU C R , WU Z S , et al. Image analysis of weld tracking based on active vision[J]. Welding Technology, 2020, 49 (2): 75-78, 6.
6
胡曦, 余震, 刘海生. 管道全位置焊缝三维测量研究[J]. 石油机械, 2021, 49 (9): 129- 136.
HU X , YU Z , LIU H S . Research on 3D measurement of all-position welding seam of pipeline[J]. China Petroleum Machinery, 2021, 49 (9): 129- 136.
7
王修文, 李志伟, 李宏伟, 等. 基于双目相机的光学高度测量系统[J]. 智能计算机与应用, 2021, 11 (2): 55- 59.
WANG X W , LI Z W , LI H W , et al. Optical height measurement system based on binocular camera[J]. Intelligent Computer and Applications, 2021, 11 (2): 55- 59.
8
许鹏飞. 基于机器视觉的焊缝三维重建及焊接飞溅物在位检测研究[D]. 济南: 山东大学, 2018.
XU P F. Three-dimensional reconstruction of weld seam and weld spatter onsite detection based on machine vision[D]. Jinan: Shandong University, 2018. (in Chinese)
9
SHAO W J , HUANG Y , ZHANG Y . A novel weld seam detection method for space weld seam of narrow butt joint in laser welding[J]. Optics & Laser Technology, 2018, 99, 39- 51.
10
WHITE R A , SMITH J S , LUCAS J . Vision-based gauge for online weld profile metrology[J]. IEE Proceedings (Science Measurement and Technology), 1994, 141 (6): 521- 526.
11
郑鹭斌, 王晓栋, 严菲. 一种基于线结构光的焊缝三维重建方法[J]. 激光与光电子学进展, 2014, 51 (4): 041005.
ZHENG L B , WANG X D , YAN F . 3D reconstruction method based on linear-structured light stripe for welding seam[J]. Laser & Optoelectronics Progress, 2014, 51 (4): 041005.
12
韩家杰, 周建平, 薛瑞雷, 等. 线结构光管道焊缝表面形貌重建与质量评估[J]. 中国激光, 2021, 48 (14): 1402010.
HAN J J , ZHOU J P , XUE R L , et al. Surface morphology reconstruction and quality evaluation of pipeline weld based on line structured light[J]. Chinese Journal of Lasers, 2021, 48 (14): 1402010.
13
李朋超, 王金涛, 宋吉来, 等. 基于线结构光扫描的复杂曲面焊缝检测[J]. 激光与光电子学进展, 2021, 58 (3): 0312005.
LI P C , WANG J T , SONG J L , et al. Weld recognition of complex curved surface based on linear structured light scanning[J]. Laser & Optoelectronics Progress, 2021, 58 (3): 0312005.
14
王克鸿, 杨嘉佳, 孙科. 基于视觉的焊接三维重建技术研究现状[J]. 机械制造与自动化, 2013, 41 (1): 1-5, 58.
WANG K H , YANG J J , SUN K . The research status of visual-based three-dimensional reconstruction technology in welding[J]. Machine Building & Automation, 2013, 41 (1): 1-5, 58.
15
洪磊, 杨小兰, 钟冬平. 基于斜率分析法的焊缝条纹直线特征提取分析[J]. 焊接学报, 2017, 38 (8): 91- 94.
HONG L , YANG X L , ZHONG D P . Feature extraction and analysis of weld seam stripe line on slope analysis method[J]. Transactions of the China Welding Institution, 2017, 38 (8): 91- 94.
16
申俊琦, 胡绳荪, 冯胜强. 激光视觉焊缝跟踪中图像二值化处理[J]. 天津大学学报, 2011, 44 (4): 308- 312.
SHEN J Q , HU S S , FENG S Q . Image binarization processing in laser vision seam tracking[J]. Journal of Tianjin University, 2011, 44 (4): 308- 312.
17
王兴东, 杨雅伦, 孔建益, 等. 基于区域优化的等厚对接焊缝图像二值化方法[J]. 中国机械工程, 2019, 30 (14): 1756- 1763.
WANG X D , YANG Y L , KONG J Y , et al. Image binarization method of equal-thickness butt welds based on regional optimization[J]. China Mechanical Engineering, 2019, 30 (14): 1756- 1763.
18
PANG S A, YANG H. An algorithm for extracting the center of linear structured light fringe based on directional template[C]//Proceedings of the 2021 4th International Conference on Advanced Electronic Materials, Computers and Software Engineering. Changsha, China: IEEE Press, 2021: 203-207.
19
LI L Y, FU L J, ZHOU X, et al. Image processing of seam tracking system using laser vision[M]//TARN T J, CHEN S B, ZHOU C J. Robotic welding, intelligence and automation. Berlin, Heidelberg: Springer, 2007: 319-324.
20
GU W P , XIONG Z Y , WAN W . Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor[J]. The International Journal of Advanced Manufacturing Technology, 2013, 69 (1-4): 451- 460.
21
ZOU Y B , CHEN T . Laser vision seam tracking system based on image processing and continuous convolution operator tracker[J]. Optics and Lasers in Engineering, 2018, 105, 141- 149.
22
SHI Y H, WANG G R, LI G J. Adaptive robotic welding system using laser vision sensing for underwater engineering[C]//Proceedings of the 2007 IEEE International Conference on Control and Automation. Guangzhou, China: IEEE Press, 2007: 1213-1218.
23
YE H M, LIU Y Z, LIU W J. Weld seam tracking based on laser imaging binary image preprocessing[C]//Proceedings of the 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference. Piscataway, USA: IEEE Press, 2021: 756-760.
24
WU Q Q , LEE J P , PARK M H , et al. A study on development of optimal noise filter algorithm for laser vision system in GMA welding[J]. Procedia Engineering, 2014, 97, 819- 827.
25
ZOU Y B , WANG Y B , ZHOU W L , et al. Real-time seam tracking control system based on line laser visions[J]. Optics & Laser Technology, 2018, 103, 182- 192.
26
HE Y S , XU Y L , CHEN Y X , et al. Weld seam profile detection and feature point extraction for multi-pass route planning based on visual attention model[J]. Robotics and Computer-Integrated Manufacturing, 2016, 37, 251- 261.
27
陈凯, 王海. 基于深度学习的焊缝图像识别研究[J]. 安徽工程大学学报, 2022, 37 (1): 24- 31.
CHEN K , WANG H . Research on weld image recognition based on deep learning[J]. Journal of Anhui Polytechnic University, 2022, 37 (1): 24- 31.
28
杭小虎, 王海. 基于深度学习的焊缝图像识别方法研究[J]. 无线互联科技, 2023, 20 (24): 126- 132.
HANG X H , WANG H . Research on weld image recognition method based on deep learning[J]. Wireless Internet Science and Technology, 2023, 20 (24): 126- 132.
29
邓贤东, 刘春华, 陈晓辉, 等. 基于深度学习的焊缝视觉跟踪方法研究[J]. 现代制造工程, 2023 (6): 124- 131.
DENG X D , LIU C H , CHEN X H , et al. Research on weld visual tracking method based on deep learning[J]. Modern Manufacturing Engineering, 2023 (6): 124- 131.
30
成冲, 章进强, 代杰. 基于卷积神经网络的不锈钢焊缝视觉检测系统[J]. 工业控制计算机, 2019, 32 (5): 51-52, 55.
CHENG C , ZHANG J Q , DAI J . A visual detection system for welding seam of stainless steel using convolutional neural network[J]. Industrial Control Computer, 2019, 32 (5): 51-52, 55.
31
WU K X , WANG T Q , HE J J , et al. Autonomous seam recognition and feature extraction for multi-pass welding based on laser stripe edge guidance network[J]. The International Journal of Advanced Manufacturing Technology, 2020, 111 (9-10): 2719- 2731.
32
ZHAO Z , LUO J , WANG Y Y , et al. Additive seam tracking technology based on laser vision[J]. The International Journal of Advanced Manufacturing Technology, 2021, 116 (1-2): 197- 211.
33
ZOU Y B , WANG Y Y . Robotic seam tracking system combining lightweight segmentation network design and ADMM-based structured pruning[J]. IEEE Transactions on Instrumentation and Measurement, 2024, 73, 5016515.
34
CHENG J M , JIN H , QIAN X D . Real-time weld seam feature extraction in construction sites[J]. Automation in Construction, 2024, 160, 105330.
35
CHEN X Y , FANG C , HU A L , et al. A novel reflective interference mitigation model for laser stripe extraction[J]. Measurement, 2024, 237, 115187.
36
WANG B , LI F S , LU R J , et al. Weld feature extraction based on semantic segmentation network[J]. Sensors, 2022, 22 (11): 4130.
37
CHEN J , WANG C C , SHI F , et al. DSNet: A dynamic squeeze network for real-time weld seam image segmentation[J]. Engineering Applications of Artificial Intelligence, 2024, 133, 108278.
38
HE Y S , CAI R , DAI F L , et al. A unified framework based on semantic segmentation for extraction of weld seam profiles with typical joints[J]. Journal of Manufacturing Processes, 2024, 131, 2275- 2287.
39
RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]//Proceedings of the Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015). Munich, Germany: Springer, 2015: 234-241.
40
ZHANG Z . A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 22 (11): 1330- 1334.
41
XIE E Z, WANG W H, YU Z D, et al. SegFormer: Simple and efficient design for semantic segmentation with transformers[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook, USA: Curran Associates, 2021: 12077-12090.

RIGHTS & PERMISSIONS

All rights reserved. Unauthorized reproduction is prohibited.
PDF(5795 KB)

Accesses

Citation

Detail

Sections
Recommended

/