Precise multi-dimensional features positioning of Xianglushan tunnel drilling based on deep-machine vision
WANG Xu1,2, GONG Xiaowen3, HUANG Qishuai3, CHEN Bingrui1,2, YANG Shiqiang4, YANG Xu5, ZHANG Yanjie6
1. State Key Laboratory of Geomechanics and Geotechnical Engineering, Institute of Rock and Soil Mechanics, Chinese Academy of Sciences, Wuhan 430071, China; 2. University of Chinese Academy of Sciences, Beijing 100049, China; 3. Yunnan Dianzhong Water Diversion Phase II Engineering Co., Ltd., Kunming 650000, China; 4. Changjiang River Scientific Research Institute, Wuhan 430010, China; 5. Yunnan Qujing Fuyuan Administration of Water Resources, Qujing 655500, China; 6. Yunnan Dianzhong Water Diversion Engineering Co., Ltd., Kunming 650000, China
Abstract：[Objective] Xianglushan tunnel is a key project of the Yunnan Dianzhong Water Diversion Engineering, and rock bursts are one of its main hazards. The precise positioning of microseismic sensor coordinates is the basis for microseismic monitoring and early warning systems for rock bursts, which is controlled by drilling coordinates and angles. However, the drilling location needs to be measured manually, which has low accuracy and poor timeliness, seriously hindering the development of automatic microseismic monitoring systems for rock bursts. Meanwhile, the precise positioning of multi-dimensional features is the key to realizing the full automation of drilling operations, including the coordinates, diameter, plane direction, and busbar direction of drilling. This study presents a method for the precise positioning of multi-dimensional features of tunnel drilling based on deep-machine vision called YOLO-AT. [Methods] The proposed method includes two modules: drilling contour detection and drilling multi-dimensional feature positioning. The drilling contour detection module adds rotation angle prediction to the YOLO V8 model, thereby constructing a YOLO V8 OBB model that can accurately fit the drilling contour. The drilling multi-dimensional feature positioning module establishes an anchor tracking algorithm, selects anchor points on contour ellipses, and achieves anchor tracking from multiple perspectives based on epipolar line constraints and projective invariance of anchor order. The spatial coordinates of each anchor point and multi-dimensional features were solved using the parallax method and analytical geometry theory, respectively. [Results] To verify the robustness of the proposed method to contour quality, a synthetic drilling contour database with controllable quality was established for Xianglushan tunnel, and the positioning results were compared with those of the ALSR, FED, AAMED, YOLO V8, and DLT methods. Results showed that: (1) concerning contour detection accuracy, the proposed method exhibited the best performance among other methods, with the average intersection over union (IoU) ratio, average F1 score, precision, and recall at an IoU threshold corresponding to 0.9 of 0.957, 0.948, 0.977, and 0.977, respectively. (2) Concerning contour detection stability, the proposed method was less affected by contour quality and always maintained high accuracy. By contrast, YOLO V8 had less fluctuation and low indicators, whereas other methods had poor stability and high accuracy under good contour quality. (3) Concerning the accuracy of multi-dimensional feature positioning, the median errors of the proposed method for the coordinate, diameter, plane direction, and busbar direction of drilling were 0.835 mm, 0.795 mm, 0.567°, and 1.751°, respectively, which were the best among all methods. (4) Concerning the stability of multi-dimensional feature positioning, the proposed method and YOLO V8-AT had better stability. Conversely, the stability of other methods rapidly decreased as the quality of the drilling contour deteriorated. [Conclusions] The proposed YOLO-AT method can achieve accurate and stable positioning of multi-dimensional features in Xianglushan tunnel drilling, with a performance superior to that of existing methods, thereby having the potential to be extended to various underground engineering drilling positions. Next, the research intends to focus on combining the proposed method and laser ranging technology to measure hole depth and eliminate ambiguity in drilling direction, in addition to analyzing the roles of RANSAC and reprojection methods in improving algorithm accuracy and stability.
王旭, 巩晓雯, 黄其帅, 陈炳瑞, 杨世强, 杨旭, 张延杰. 基于深度机器视觉的香炉山隧洞钻孔多维特征精准定位[J]. 清华大学学报（自然科学版）, 2024, 64(7): 1278-1292.
WANG Xu, GONG Xiaowen, HUANG Qishuai, CHEN Bingrui, YANG Shiqiang, YANG Xu, ZHANG Yanjie. Precise multi-dimensional features positioning of Xianglushan tunnel drilling based on deep-machine vision. Journal of Tsinghua University(Science and Technology), 2024, 64(7): 1278-1292.
[1] 欧阳林,张如九,刘耀儒,等.深埋隧洞岩爆防控技术及典型工程应用现状综述[J].长江科学院院报, 2022, 39(12):161-170. OUYANG L, ZHANG R J, LIU Y R, et al. Review on rockburst prevention techniques and typical applications in deep tunnels[J]. Journal of Yangtze River Scientific Research Institute, 2022, 39(12):161-170.(in Chinese) [2] 陈炳瑞,冯夏庭,符启卿,等.综合集成高精度智能微震监测技术及其在深部岩石工程中的应用[J].岩土力学, 2020, 41(7):2422-2431. CHEN B R, FENG X T, FU Q Q, et al. Integration and high precision intelligence microseismic monitoring technology and its application in deep rock engineering[J]. Rock and Soil Mechanics, 2020, 41(7):2422-2431.(in Chinese) [3] CHEN B R, WANG X, ZHU X H, et al. Real-time arrival picking of rock microfracture signals based on convolutional-recurrent neural network and its engineering application[J]. Journal of Rock Mechanics and Geotechnical Engineering, 2024, 16(3):761-777. [4] 张楚旋,李夕兵,董陇军,等.微震监测传感器布设方案评价模型及应用[J].东北大学学报(自然科学版), 2016, 37(4):594-598, 608. ZHANG C X, LI X B, DONG L J, et al. Evaluation model of microseismic monitoring sensor layout scheme and its application[J]. Journal of Northeastern University (Science and Technology), 2016, 37(4):594-598, 608.(in Chinese) [5] 衡献伟,李青松,韩真理,等.微震监测系统传感器安装技术及应用[J].煤炭技术, 2015, 34(12):213-214. HENG X W, LI Q S, HAN Z L, et al. Mounting technology and application of microseismic monitoring system sensor[J]. Coal Technology, 2015, 34(12):213-214.(in Chinese) [6] 李涛.不良条件下微震源定位及工程应用研究[D].北京:中国科学院大学, 2022. LI T. Research on micro seismic source location and engineering application under adverse conditions[D]. Beijing:University of Chinese Academy of Sciences, 2022.(in Chinese) [7] 杨向东,芮晓飞,谢颖.基于高效Hough变换的圆柱特征检测方法[J].清华大学学报(自然科学版), 2015, 55(8):921-926. YANG X D, RUI X F, XIE Y. Efficient Hough transform-based cylinder feature detection algorithm[J]. Journal of Tsinghua University (Science and Technology), 2015, 55(8):921-926.(in Chinese) [8] MEI B, ZHU W D, YAN G R, et al. A new elliptic contour extraction method for reference hole detection in robotic drilling[J]. Pattern Analysis and Applications, 2015, 18(3):695-712. [9] PRASAD D K, LEUNG M K H, CHO S Y. Edge curvature and convexity based ellipse detection method[J]. Pattern Recognition, 2012, 45(9):3204-3221. [10] 贾棋,梁景朝,王祎,等.基于区域检测和弧筛选的椭圆检测方法[J].计算机辅助设计与图形学学报, 2022, 34(11):1784-1794. JIA Q, LIANG J C, WANG Y, et al. Ellipse detection combining region detection and arc filtering[J]. Journal of Computer-Aided Design&Computer Graphics, 2022, 34(11):1784-1794.(in Chinese) [11] 范安然.基于双目立体视觉的道岔钢轨钻孔的尺寸测量[D].秦皇岛:燕山大学, 2015. FAN A R. Size measurement of the switch rail drilling based on binocular stereo vision[D]. Qinhuangdao:Yanshan University, 2015.(in Chinese) [12] 胡彦强,马钺,许敏.白车身孔槽类特征三维坐标在线测量方法研究[J].计算机工程, 2017, 43(10):186-191, 197. HU Y Q, MA Y, XU M. Study of on-line measurement method for 3D coordinates of body-in-white hole slot features[J]. Computer Engineering, 2017, 43(10):186-191, 197.(in Chinese) [13] LEI M Y, ZHANG X H, DONG Z, et al. Locating anchor drilling holes based on binocular vision in coal mine roadways[J]. Mathematics, 2023, 11(20):4365. [14] 黄金.基于双目视觉的机械臂锚护孔位精准定位技术研究[D].太原:太原科技大学, 2022. HUANG J. Research on recise positioning technology of manipulator anchor hole based on binocular vision[D].Taiyuan:Taiyuan University of Science and Technology, 2022.(in Chinese) [15] 魏振忠,张广军.视觉检测中椭圆中心成像畸变误差模型研究[J].北京航空航天大学学报, 2003, 29(2):140-143. WEI Z Z, ZHANG G J. Distortion error model of image of ellipse center in 3D visual inspection[J]. Journal of Beijing University of Aeronautics and Astronautics, 2003, 29(2):140-143.(in Chinese) [16] 王琳毅,白静,李文静,等. YOLO系列目标检测算法研究进展[J].计算机工程与应用, 2023, 59(14):15-29. WANG L Y, BAI J, LI W J, et al. Research progress of YOLO series target detection algorithms[J]. Computer Engineering and Applications, 2023, 59(14):15-29.(in Chinese) [17] 方涛涛,王池社,王洁,等.基于YOLO v8n的探地雷达图像管线定位方法[J].国外电子测量技术, 2023, 42(11):170-177. FANG T T, WANG C S, WANG J, et al. Ground penetrating radar image pipeline location based on YOLO v8n[J]. Foreign Electronic Measurement Technology, 2023, 42(11):170-177.(in Chinese) [18] 宋怀波,焦义涛,华志新,等.基于YOLO v5-OBB与CT的浸种玉米胚乳裂纹检测[J].农业机械学报, 2023, 54(3):394-401, 439. SONG H B, JIAO Y T, HUA Z X, et al. Endosperm crack detection method for seed dipping maize based on YOLO v5-OBB and CT technology[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(3):394-401, 439.(in Chinese) [19] 蒋弘毅,王永娟,康锦煜.目标检测模型及其优化方法综述[J].自动化学报, 2021, 47(6):1232-1255. JIANG H Y, WANG Y J, KANG J Y. A survey of object detection models and its optimization methods[J]. Acta Automatica Sinica, 2021, 47(6):1232-1255.(in Chinese) [20] HOU L P, LU K, YANG X, et al. G-Rep:Gaussian representation for arbitrary-oriented object detection[J]. Remote Sensing, 2023, 15(3):757. [21] MURRUGARRA-LLERENA J, KIRSTEN L N, ZENI L F, et al. Probabilistic intersection-over-union for training and evaluation of oriented object detectors[J]. IEEE Transactions on Image Processing, 2024, 33:671-681. [22] NEUBECK A, VAN GOOL L. Efficient non-maximum suppression[C]//Proceedings of the 18th International Conference on Pattern Recognition (ICPR 2006). Hong Kong, China:IEEE, 2006:850-855. [23] MILLER J R, GOLDMAN R N. Using tangent balls to find plane sections of natural quadrics[J]. IEEE Computer Graphics and Applications, 1992, 12(2):68-82. [24] LU C S, XIA S Y, SHAO M, et al. Arc-support line segments revisited:An efficient high-quality ellipse detection[J]. IEEE Transactions on Image Processing, 2020, 29:768-781. [25] JIA Q, FAN X, LUO Z X, et al. A fast ellipse detector using projective invariant pruning[J]. IEEE Transactions on Image Processing, 2017, 26(8):3665-3679. [26] MENG C, LI Z X, BAI X Z, et al. Arc adjacency matrix-based fast ellipse detection[J]. IEEE Transactions on Image Processing, 2020, 29:4406-4420. [27] 王斌,吴丹,盖宇航.面向机器人精密装配的高精度圆片位姿视觉检测[J].机械工程学报, 2023, 59(8):50-59. WANG B, WU D, GAI Y H. High-precision wafer pose visual detection for robot precise assembly[J]. Journal of Mechanical Engineering, 2023, 59(8):50-59.(in Chinese) [28] WU W Q, WANG X G, XU D, et al. Position and orientation measurement for autonomous aerial refueling based on monocular vision[J]. International Journal of Robotics and Automation, 2017, 32(1):13-21. [29] 赵祚喜,冯荣,朱裕昌,等.空间点的多视图DLT三维定位[J].光学精密工程, 2020, 28(1):212-222. ZHAO Z X, FENG R, ZHU Y C, et al. Multi-view DLT three-dimensional positioning method for spatial points[J]. Optics and Precision Engineering, 2020, 28(1):212-222.(in Chinese)
2024, Vol. 64 
