Special automatic spraying system for civil aircraft parts based on visual recognition
JIANG Shuai1, SONG Libin2, CHEN Xiaoyong2, ZHANG Peng2, LIU Kecheng3, CHANG Junhu2
1. Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China; 2. Research Center for Robot and Automation Equipment, Tianjin Research Institute for Advanced Equipment, Tsinghua University, Tianjin 300300, China; 3. AVIC SAC Commercial Aircraft Company Limited, Shenyang 110169, China
Abstract：[Objective] For a long time, manual operation is the only option for spraying aircraft parts. Uncertainty in a manual operation leads to problems including uneven paint film thickness, low spraying efficiency, and significant paint waste. More than 50 000 types of parts are available for various civil aircraft. With a wide variety, no fixed placement, and no clamping device, all these problems are far from being solved using existing robotic systems. To target the problems, an automatic spraying system for civil aircraft parts based on visual recognition is developed. [Methods] The system combines a two-dimensional (2D) camera and a three-dimensional (3D) camera to scan the spraying platform to realize the recognition and pose matching of parts. The 2D camera is used to obtain a 2D image of the parts on the spraying platform. Parts are preliminarily located and classified using the images, thereby reducing the interference of the environment on the point cloud. The 3D camera simply scans the partial space marked using the 2D camera, which effectively shortens the time compared to scanning the entire spraying platform space. The viewpoint feature histogram (VFH) descriptor is used to describe the point cloud features to recognize the type of parts. The k-dimensional tree (kd-tree) is used to establish the feature index. The search performance is much better than the global nearest neighbor search method. To solve the problem that the particle swarm optimization (PSO) algorithm is easy to fall into local optimal extremum, an improved PSO algorithm is proposed, where stagnation coefficient and the concept of decoupling are combined with the proposed algorithm. The spraying trajectory of the robot is planned, including the path of the spray gun and the trajectory of the robotic arm joints. The spray gun path planning means planning the path that the spray gun takes in the Cartesian space. Spraying paths are planned using the surface slicing method to realize the full coverage spraying of random poses and complex surface parts. Joint trajectory planning means planning the angle trajectory of each joint of the robot. The Bézier curve is used to plan the joint space trajectory to ensure the robot operated smoothly. [Results] The improved PSO algorithm performed better than the traditional PSO algorithm in convergence speed and accuracy. With an average accuracy of 2.11 mm, the automated point cloud registration for complex parts was completed in 240 s. Spray paths were simulated with Robot Studio, and simulations verified the method's effectiveness. [Conclusions] The spraying issue of numerous civil aircraft parts at arbitrary placements is resolved using the robot automatic spraying system based on visual recognition described in this study. Work efficiency and spraying quality of the system meet the production requirements. It is also found that the ambient light interferes with the work of segmenting parts in 2D images during the experiment. Manual adjustment of parameters cannot produce good results for complex image analysis tasks with high noise and shadow, so an adaptive parameter method should be proposed. The greedy algorithm is used for spraying path combinations, and the paths' global optimization method needs to be improved.
姜帅, 宋立滨, 陈晓永, 张朋, 刘科成, 常俊虎. 基于视觉识别的民机零件专用自动喷涂系统[J]. 清华大学学报（自然科学版）, 2023, 63(10): 1650-1657.
JIANG Shuai, SONG Libin, CHEN Xiaoyong, ZHANG Peng, LIU Kecheng, CHANG Junhu. Special automatic spraying system for civil aircraft parts based on visual recognition. Journal of Tsinghua University(Science and Technology), 2023, 63(10): 1650-1657.
[1] 宋袁曾,陈洁,毛景.大型飞机整机涂装自动化实施探讨与展望[J].航空制造技术, 2016(10):52-56. SONG Y Z, CHEN J, MAO J. Discussion and prospects of trunk aircraft exterior automatic painting for large civil aircraft[J]. Aeronautical Manufacturing Technology, 2016(10):52-56.(in Chinese) [2] 王国磊,王宁涛,陈恳.面向整机的机器人喷涂系统回顾与展望[J].航空制造技术, 2016(16):76-80. WANG G L, WANG N T, CHEN K. Review and prospects of robotic aircraft painting system[J]. Aeronautical Manufacturing Technology, 2016(16):76-80.(in Chinese) [3] 冯华山,秦现生,王润孝.航空航天制造领域工业机器人发展趋势[J].航空制造技术, 2013(19):32-37. FENG H S, QIN X S, WANG R X. Developing trend of industrial robot in aerospace manufacturing industry[J]. Aeronautical Manufacturing Technology, 2013(19):32-37.(in Chinese) [4] ANANDAN T M. Aerospace manufacturing on board with robots[Z/OL].(2016-02-18)[2022-05-15] . https://www.automate.org/industry-insights/aerospace-manufact-uringon-board-with-robots. [5] SEEGMILLER N A, BAILIFF J A, FRANKS R K. Precision robotic coating application and thickness control optimization for F-35 final finishes[J]. SAE International Journal of Aerospace, 2009, 2(1):284-290. [6] 曹玉满.航空紧固件自动喷涂机控制系统的开发[D].重庆:重庆大学, 2013. CAO Y M. The design of control system of aerospace fastener automatic spraying machine[D]. Chongqing, China:Chongqing University, 2013.(in Chinese) [7] 缪东晶,吴聊,徐静,等.飞机表面自动喷涂机器人系统与喷涂作业规划[J].吉林大学学报(工学版), 2015, 45(2):547-553. MIAO D J, WU L, XU J, et al. Automatic spraying robot system for aircraft surfaces and spraying operation planning[J]. Journal of Jilin University (Engineering and Technology Edition), 2015, 45(2):547-553.(in Chinese) [8] 朱利中.飞机大部件喷涂机器人的设计与分析[D].成都:电子科技大学, 2015. ZHU L Z. Design and analysis of a spraying robot for large-dimension aircraft components[D]. Chengdu, China:University of Electronic Science and Technology of China, 2015.(in Chinese) [9] 赵继,赵军,张雷,等.焊缝磨抛机器人视觉算法实现及其试验研究[J].机械工程学报, 2013, 49(20):42-48. ZHAO J, ZHAO J, ZHANG L, et al. Vision algorithm and experimental study on the robotic weld-bead grinding and polishing system[J]. Journal of Mechanical Engineering, 2013, 49(20):42-48.(in Chinese) [10] 徐超凡.基于立体光源的工业外观缺陷检测平台的算法设计与实现[D].哈尔滨:哈尔滨工业大学, 2019. XU C F. Algorithm design and implementation of industrial appearance defect detection platform based on stereoscopic light source[D]. Harbin, China:Harbin Institute of Technology, 2019.(in Chinese) [11] HALBER M, FUNKHOUSER T. Fine-to-coarse global registration of RGB-D scans[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA:IEEE, 2017:6660-6669. [12] RUSU R B, BRADSKI G, THIBAUX R, et al. Fast 3D recognition and pose using the viewpoint feature histogram[C]//2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China:IEEE, 2010:2155-2162. [13] 杜振鹏,李德华.基于kd-tree搜索和SURF特征的图像匹配算法研究[J].计算机与数字工程, 2012, 40(2):96-98, 126. DU Z P, LI D H. Image matching algorithm research based on kd-tree search and SURF features[J]. Computer and Digital Engineering, 2012, 40(2):96-98, 126.(in Chinese) [14] ZHU J H, ZHU L, JIANG Z T, et al. Local to global registration of multi-view range scans using spanning tree[J]. Computers&Electrical Engineering, 2016, 58:477-488. [15] GUO R, ZHU J H, LI Y C, et al. Weighted motion averaging for the registration of multi-view range scans[J]. Multimedia Tools and Applications, 2018, 77(9):10651-10668. [16] GE Y Q, WANG B Y, NIE J H, et al. A point cloud registration method combining enhanced particle swarm optimization and iterative closest point method[C]//2016 Chinese Control and Decision Conference. Yinchuan, China:IEEE, 2016:2810-2815. [17] SHENG W H, CHEN H P, XI N, et al. Tool path planning for compound surfaces in spray forming processes[J]. IEEE Transactions on Automation Science and Engineering, 2005, 2(3):240-249. [18] SHENG W H, XI N, CHEN H P, et al. Surface partitioning in automated CAD-guided tool planning for additive manufacturing[C]//Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, USA:IEEE, 2003:2072-2077. [19] CAI Z H, DENG S H, LIAO H L, et al. The effect of spray distance and scanning step on the coating thickness uniformity in cold spray process[J]. Journal of Thermal Spray Technology, 2014, 23(3):354-362. [20] MA J W, LIU Y, ZANG S F, et al. Robot path planning based on genetic algorithm fused with continuous Bézier optimization[J]. Computational Intelligence and Neuroscience, 2020, 2020:9813040. [21] ZHANG L S, SUN L, ZHANG S, et al. Trajectory planning for an indoor mobile robot using quintic Bezier curves[C]//2015 IEEE International Conference on Robotics and Biomimetics. Zhuhai, China:IEEE, 2015:757-762. [22] 余伶俐,龙子威,周开军.基于贝塞尔曲线的机器人非时间轨迹跟踪方法[J].仪器仪表学报, 2016, 37(7):1564-1572. YU L L, LONG Z W, ZHOU K J. Non-time trajectory tracking method based on Bézier curve for robot[J]. Chinese Journal of Scientific Instrument, 2016, 37(7):1564-1572.(in Chinese)
2023, Vol. 63 
