Robotic hole drilling attitude correction method based on structured light 3-D visual measurements
CHEN Lu1, GUAN Liwen1, LIU Chun2, CHEN Zhixiong3, XUE Jun4
1. Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China; 2. Chengdu Aircraft Industrial(Group) Co., Ltd., Chengdu 610092, China; 3. School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China; 4. AVIC Manufacturing Technology Institute, Beijing 100025, China
Abstract：The accuracy of the normal direction during hole drilling greatly affects the assembly accuracy and the service life and safety of aircraft. Therefore, better methods are needed for pose correction based on advanced visual detection methods. A reference normal direction detection method was developed for robotic hole drilling using a robotic visual inspection system that was tested using simulations and a prototype. The systems used a 3-D measurement model of the normal vector detection module to generate a point cloud on the workpiece surface. The point cloud in the camera coordinate system was then transformed into a point cloud in the hole drilling tool coordinate system by an online hand-eye calibration method based on a untraceless Kalman filter. After the normal vector to the surface is identified, a robotic hole drilling attitude correction algorithm is used to guide the drill. 3-D measurements show that this normal vector detection method is accurate and meets the hole specification requirements for the aviation industry.
[1] 周华飞. 机器人自动化制孔中位姿误差的修正与补偿[D]. 杭州:浙江大学, 2015. ZHOU H F. Correction and compensation for positioning- orientation error of robot in automatic drilling[D]. Hangzhou:Zhejiang University, 2015. (in Chinese) [2] 薛汉杰, 张敬佩. 蒙皮类部件钻孔法向的测量和调整[J]. 航空制造技术, 2010(23):60-62, 66. XUE H J, ZHANG J P. Normal measurement and adjustment for skin drilling[J]. Aeronautical Manufacturing Technology, 2010(23):60-62, 66. (in Chinese) [3] ZHU W D, MEI B, YAN G R, et al. Measurement error analysis and accuracy enhancement of 2D vision system for robotic drilling[J]. Robotics and Computer-Integrated Manufacturing, 2014, 30(2):160-171. [4] 公茂震, 袁培江, 王田苗, 等. 航空制孔机器人末端垂直度智能调节方法[J]. 北京航空航天大学学报, 2012, 38(10):1400-1404. GONG M Z, YUAN P J, WANG T M, et al. Intelligent verticality-adjustment method of end-effector in aeronautical drilling robot[J]. Journal of Beijing University of Aeronautics and Astronautics, 2012, 38(10):1400-1404. (in Chinese) [5] TIAN W, ZHOU W X, ZHOU W, et al. Auto-normalization algorithm for robotic precision drilling system in aircraft component assembly[J]. Chinese Journal of Aeronautics, 2013, 26(2):495-500. [6] SONG T, XI F F, GUO S, et al. A comparison study of algorithms for surface normal determination based on point cloud data[J]. Precision Engineering, 2015, 39:47-55. [7] 张诚然. 机身蒙皮自动铆接的辅助定位视觉伺服系统设计[D]. 沈阳:沈阳航空航天大学, 2018. ZHANG C R. Design of auxiliary positioning visual servo system for automatic riveting to fuselage panel[D]. Shenyang:Shenyang Aerospace University, 2018. (in Chinese) [8] EGUTI C C A, TRABASSO L G. Design of a robotic orbital driller for assembling aircraft structures[J]. Mechatronics, 2014, 24(5):533-545. [9] ZHANG R, YUAN P J, GONG M Z. Intelligent surface-normal adjustment system and application in aeronautical drilling robot[C]//2012 Second International Conference on Intelligent System Design and Engineering Application. Sanya, China:IEEE, 2012:696-699. [10] 张为民. 基于视觉信息处理的飞行器型面法向矢量的在线测量方法[J]. 航空制造技术, 2015(19):43-46, 87. ZHANG W M. Aircraft surface normal vector on line measuring method base on visual information processing[J]. Aeronautical Manufacturing Technology, 2015(19):43-46, 87. (in Chinese) [11] RAO G, WANG G L, YANG X D, et al. Normal direction measurement and optimization with a dense three-dimensional point cloud in robotic drilling[J]. IEEE/ASME Transactions on Mechatronics, 2018, 23(3):986-996. [12] RAO G, YANG X D, XU J, et al. Normal direction measurement based on 3D point cloud in robotic drilling[C]//2017 IEEE 7th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). Honolulu, USA:IEEE, 2017:1182-1187. [13] 饶刚. 机器人制孔法矢的面结构光三维测量与控制研究[D]. 北京:清华大学, 2019. RAO G. Research on 3D measurement and control of surface structured light for robot drilling normal vector[D]. Beijing:Tsinghua University, 2019. (in Chinese) [14] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334. [15] 陈彦军, 左旺孟, 王宽全, 等. 结构光编码方法综述[J]. 小型微型计算机系统, 2010, 31(9):1856-1863. CHEN Y J, ZUO W M, WANG K Q, et al. Survey on structured light pattern codification methods[J]. Journal of Chinese Computer Systems, 2010, 31(9):1856-1863. (in Chinese) [16] 张广军, 王红, 赵慧洁, 等. 结构光三维视觉系统研究[J]. 航空学报, 1999, 20(4):365-367. ZHANG G J, WANG H, ZHAO H J, et al. Structured light 3D vision[J]. Acta Aeronauticaet Astronautica Sinica, 1999, 20(4):365-367. (in Chinese) [17] ZHANG H, BERG A C, MAIRE M, et al. SVM-KNN:Discriminative nearest neighbor classification for visual category recognition[C]//2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). New York, USA:IEEE, 2006:2126-2136. [18] 王君臣, 王田苗, 杨艳, 等. 基于无迹卡尔曼滤波的机器人手眼标定[J]. 机器人, 2011, 33(5):621-627. WANG J C, WANG T M, YANG Y, et al. Robot hand-eye calibration using unscented Kalman filtering[J]. Robot, 2011, 33(5):621-627. (in Chinese)
2022, Vol. 62 
