[Objective] Oblong holes are commonly used across various industries to improve fault tolerance and adjustment capabilities. However, their complex geometric characteristics pose significant challenges for vision detection and location algorithms in industrial applications, impacting their utilization in automatic assembly processes. [Methods] This research investigates a high-precision and robust vision segmentation and location algorithm tailored for oblong holes. First, the geometric features of oblong holes, which are symmetric but lack a simple analytical description, are analyzed. This complexity renders traditional imaging methods ineffective for accurate localization. The detection and segmentation of oblong hole features are conducted using a novel vision location algorithm that integrates deep learning with conventional image processing techniques. Specifically, the algorithm employs a sequential connection framework of YOLO and fully convolutional networks to achieve accurate localization. This framework first identifies the region of interest and then performs semantic segmentation. YOLO networks rapidly detect the region of interest, prioritizing areas where the oblong hole is prominently featured. Semantic segmentation is subsequently performed using fully convolutional networks. Afterward, a skeleton feature extraction method based on medial axis transformation is applied to precisely locate the oblong hole. This method effectively reduces the impact of shape errors from semantic segmentation, achieving subpixel accuracy. However, medial axis transformation may produce redundant lines owing to the presence of image artifacts, potentially leading to inaccuracies. To address this issue, principal component analysis is employed to approximate the center of the oblong hole, thereby minimizing errors. For further precision, a Hough transformation ellipse detection method is utilized to identify the central skeleton of the oblong hole, which is interpreted both as a line segment and a special ellipse. The center of this skeleton represents the center of the oblong hole. [Results] Experimental validation conducted in a specific robotics automatic assembly system confirms the effectiveness of the proposed algorithm. The robustness of the algorithm is further demonstrated through image sampling using camera hardware distinct from that used in the training dataset. Additionally, the impact of surface features and oblong hole shapes on the detection performance is analyzed. The experimental outcomes indicate the optimal performance of the algorithm on objects with nonreflective surfaces, with minimal effect from the shape of the oblong hole on accuracy. Despite potential deformations in segmentation output due to hardware variations, the oblong hole region degenerating location algorithm, based on medial axis transformation, accurately locates the center. The final location error is recorded at 1.05 pixels, which surpasses the accuracy achieved through the direct calculation of the center of gravity of the segmented region. These results underscore the substantial benefits of the algorithm in scenarios with varying hardware and object conditions, demonstrating its high accuracy and exceptional robustness. [Conclusions] By merging deep learning techniques with traditional image processing methods, the location tasks for diverse objects are effectively resolved. The extraction of highly nonlinear features through deep learning, followed by processing with traditional image methods incorporating prior geometric knowledge, enhances the robustness and accuracy of the algorithm, making it suitable for practical production applications.
JIANG Xiao
,
WANG Song
,
WU Dan
. Segmentation and location algorithm for oblong holes in robotic automatic assembly[J]. Journal of Tsinghua University(Science and Technology), 2024
, 64(10)
: 1677
-1685
.
DOI: 10.16511/j.cnki.qhdxxb.2024.27.023
[1] AKINLAR C, TOPAL C. EDCircles: A real-time circle detector with a false detection control [J]. Pattern Recognition, 2013, 46(3): 725-740.
[2] MUKHOPADHYAY P, CHAUDHURI B B. A survey of Hough transform [J]. Pattern Recognition, 2015, 48(3): 993-1010.
[3] TOPAL C, AKINLAR C. Edge Drawing: A combined real-time edge and segment detector [J]. Journal of Visual Communication and Image Representation, 2012, 23(6): 862-872.
[4] ZHOU B, HE Y. Fast circle detection using spatial decomposition of Hough transform [J]. International Journal of Pattern Recognition and Artificial Intelligence, 2017, 31(3): 1755006.
[5] 刘艳霞. 封闭图形对称轴的研究与应用[D]. 长春: 吉林大学, 2015. LIU Y X. Symmetry axis research and application for closed graph [D]. Changchun: Jilin University, 2015. (in Chinese)
[6] 李晓磊, 潘晋孝, 刘宾, 等. 基于Hough算法的图像对称轴检测(英文) [J]. 测试科学与仪器, 2015, 6(4): 342-346. LI X L, PAN J X, LIU B, et al. Symmetric axis detection for images based on Hough algorithm [J]. Journal of Measurement Science and Instrumentation, 2015, 6(4): 342-346.
[7] SUZUKI S, BE K. Topological structural analysis of digitized binary images by border following [J]. Computer Vision, Graphics, and Image Processing, 1985, 30(1): 32-46.
[8] 彭正初. 基于傅里叶描述子的物体形状识别的研究[D]. 哈尔滨: 哈尔滨工业大学, 2016. PENG Z C. Research on object shape recognition based on Fourier descriptors [D]. Harbin: Harbin Institute of Technology, 2016. (in Chinese)
[9] LOWE D G. Object recognition from local scale-invariant features [C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece: IEEE, 1999: 1150-1157.
[10] 刘立, 詹茵茵, 罗扬, 等. 尺度不变特征变换算子综述[J]. 中国图象图形学报, 2013, 18(8): 885-892. LIU L, ZHAN Y Y, LUO Y, et al. Summarization of the scale invariant feature transform [J]. Journal of Image and Graphics, 2013, 18(8): 885-892. (in Chinese)
[11] 刘强, 段富海, 桑勇, 等. 复杂环境下视觉SLAM闭环检测方法综述[J]. 机器人, 2019, 41(1): 112-123, 136. LIU Q, DUAN F H, SANG Y, et al. A survey of loop-closure detection method of visual SLAM in complex environments [J]. Robot, 2019, 41(1): 112-123, 136. (in Chinese)
[12] LONG C R, LIU P, HU Q L. Ellipse fitting based on neural network optimization [C]//202241st Chinese Control Conference (CCC). Hefei, China: IEEE, 2022: 467-471.
[13] MINAEE S, BOYKOV Y, PORIKLI F, et al. Image segmentation using deep learning: A survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3523-3542.
[14] 王斌. 一种基于多级弦长函数的傅立叶形状描述子[J]. 计算机学报, 2010, 33(12): 2387-2396. WANG B. A Fourier shape descriptor based on multi-level chord length function [J]. Chinese Journal of Computers, 2010, 33(12): 2387-2396. (in Chinese)
[15] 陈海峰. 数字图像中基本几何形状检测算法的研究与应用[D]. 杭州: 浙江大学, 2007. CHEN H F. Research on the basic geometric shapes detection in digital images and the application [D]. Hangzhou: Zhejiang University, 2007. (in Chinese)
[16] HOUGH P V C. Method and means for recognizing complex patterns: US3069654[P]. 1962-12-18.
[17] FISCHLER M A, BOLLES R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM, 1981, 24(6): 381-395.
[18] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016: 779-788.
[19] LONG J, SHELHAMER E, DARRELL T, et al. Fully convolutional networks for semantic segmentation [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015: 3431-3440.
[20] LEE D T. Medial axis transformation of a planar shape [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1982, PAMI-4(4): 363-369.
[21] 史聪伟, 赵杰煜, 常俊生. 基于中轴变换的骨架特征提取算法[J]. 计算机工程, 2019, 45(7): 242-250. SHI C W, ZHAO J Y, CHANG J S. Skeleton feature extraction algorithm based on medial axis transformation [J]. Computer Engineering, 2019, 45(7): 242-250. (in Chinese)
[22] GARRIDO-JURADO S, MUÑOZ-SALINAS R, MADRID- CUEVAS F J, et al. Automatic generation and detection of highly reliable fiducial markers under occlusion [J]. Pattern Recognition, 2014, 47(6): 2280-2292.
[23] GARRIDO-JURADO S, MUÑOZ-SALINAS R, MADRID- CUEVAS F J, et al. Generation of fiducial marker dictionaries using Mixed Integer Linear Programming [J]. Pattern Recognition, 2016, 51: 481-491.
