Research Progress of Visual Servo Control Technology for Picking Robot
-
摘要:
在新型农业生产模式下,采摘机器人成为智能农业装备的重要组成部分。在非结构环境下,视觉伺服控制以获得作业环境信息量大、鲁棒性强等优势成为采摘机器人的关键技术。结合3种经典视觉伺服控制方法的研究进展和原理,对比总结了各种控制方法的优缺点和研究重点,并就其中一些关键问题分析了解决方法,同时提出对采摘机器人适应性强的视觉伺服控制技术,并对技术发展趋势进行了展望。
Abstract:Under new agricultural production mode, picking robot has become an important part of intelligent agricultural equipment.In unstructured environment, visual servo control has become key technology of picking robot to obtain advantages of large amount of information and strong robustness. Combining research progress and principles of three classical visual servo control methods, advantages, disadvantages and research focus of various control methods were compared and summarized, solutions to some key problems were analyzed, visual servo control technology with strong adaptability to picking robot was put forward, and development trend of the technology was prospected.
-
表 1 视觉伺服控制方法的优缺点
Table 1. Advantages and disadvantages of visual servo control methods
伺服方法 优点 缺点 基于位置的视觉伺服 直接在三维笛卡尔空间控制机器人、可得最优笛卡尔空间轨迹、相对容易实现 对标定参数误差敏感、依赖目标模型的正确性、无法保证目标一直在相机视野内 基于图像的视觉伺服 直接在二维图像空间控制机器人、不需要三维重构、对标定误差鲁棒、能较好保证目标处于相机视野内 求解雅克比矩阵计算复杂且容易进入奇异点、相机可能收敛于局部最小点、控制器设计困难 混合视觉伺服 实现旋转和平移分量的解耦、对标定误差鲁棒、不依赖目标模型、可达全局稳定 求解单应性矩阵计算量大、无法保证目标一直在相机视野内 -
[1] SHIRAI Y,INOUE H.Guiding a robot by visual feedback in assembling tasks[J].Pattern Recognition,1973,5(2):99-108. doi: 10.1016/0031-3203(73)90015-0 [2] 朱大奇,颜明重.移动机器人路径规划技术综述[J].控制与决策,2010,25(7):961-967. doi: 10.13195/j.cd.2010.07.4.zhudq.014ZHU Daqi,YAN Mingzhong.Survey on technology of mobile robot path planning[J].Control and Decision,2010,25(7):961-967. doi: 10.13195/j.cd.2010.07.4.zhudq.014 [3] 周尧.基于RealSense的采摘机器人视觉伺服控制系统设计[D].南京: 南京林业大学, 2018.ZHOU Yao.Design of vision servo control system for picking robot based on RealSense[D].Nanjing: Nanjing Forestry University, 2018. [4] CHAUMETTE F,HUTCHINSON S.Visual servo control I basic approaches[J].IEEE Robotics and Automation Magazine,2006,13(4):82-90. doi: 10.1109/MRA.2006.250573 [5] CHAUMETTE F,HUTCHINSON S.Visual servo control II advanced approaches[J].IEEE Robotics and Automation Magazine,2007,14(1):109-118. doi: 10.1109/MRA.2007.339609 [6] 杨军莉.基于工业机器人视觉伺服控制系统研究[J].自动化技术与应用,2020,39(8):66- 69,73. doi: 10.3969/j.issn.1003-7241.2020.08.016YANG Junli.Research on visual servo control system based on industrial robot[J].Techniques of Automation and Applications,2020,39(8):66- 69,73. doi: 10.3969/j.issn.1003-7241.2020.08.016 [7] 张德伟, 刘海涛, 王仙业, 等.一种足式移动机器人的混合视觉伺服控制方法[J/OL].机械科学与技术.https://doi.org/10.13433/j.cnki.1003-8728.20200527.Zhang Dewei, Liu Haitao, Wang Xianye, et al.Hybrid visual servoing control of a legged mobile robot[J/OL].Mechanical Science and Technology for Aerospace Engineering.https://doi.org/10.13433/j.cnki.1003-8728.20200527. [8] LIU Y C,HUANG T S,FAUGERAS O D.Determination of camera location from 2-D to 3-D line and point correspondences[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1990,12(1):28-37. doi: 10.1109/34.41381 [9] 邹朋朋,张滋黎,王平,等.基于共线向量与平面单应性的双目相机标定方法[J].光学学报,2017,37(11):236-244.ZOU Pengpeng,ZHANG Zili,WANG Ping,et al.Binocular camera calibration based on collinear vector and plane homography[J].Acta Optica Sinica,2017,37(11):236-244. [10] LUIS R, JULIO C, MOISES J, et al.Improve three-dimensional point localization accuracy in stereo vision systems using a novel camera calibration method[J].International Journal of Advanced Robotic Systems, 2020.DOI: 10.1177/1729881419896717. [11] 陈文艺,许洁,杨辉.利用双神经网络的相机标定方法[J].红外与激光工程,2021,50(11):284-292.CHEN Wenyi,XU Jie,YANG Hui.Camera calibration method based on double neural network[J].Infrared and Laser Engineering,2021,50(11):284-292. [12] ZHANG Zhengyou.A flexible new technique for camera calibration[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(1): 1330- 1334. [13] ZHAO Chunhui, FAN Bin, HU Jinwen, et al.Pose estimation for multi-camera systems[C]//2017 IEEE International Conference on Unmanned Systems (ICUS).DOI: 10.1109/ICUS.2017.8278403. [14] ZHANG Lijun, GAO Jian, YAN Weisheng.Visual servo control for an underwater free-floating manipulator with base motion compensation[C]//2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO).DOI: 10.1109/OCEANSKOBE.2018.8559380. [15] RUPP M, BLAGOJEVIC B, KNOLL C, et al.Approximation of covariance matrices based on matching accuracy[C]//2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).DOI: 10.1109/AIM43001.2020.9158906. [16] 刘安东, 卢威威, 俞立.基于位置的机械臂视觉伺服预测控制[J/OL].小型微型计算机系统.https://kns.cnki.net/kcms/detail/21.1106.TP.20210706.1325.023.html.LIU Andong, LU Weiwei, YU Li.Position-based visual servoing predictive control for manipulators[J/OL].Journal of Chinese Computer Systems.https://kns.cnki.net/kcms/detail/21.1106.TP.20210706.1325.023.html. [17] 王亚.基于视觉反馈的机械臂控制系统研究与设计[D].哈尔滨: 哈尔滨工业大学, 2015.WANG Ya.Research and design of manipulator control system based on the visual feedback[D].Harbin: Harbin Institute of Technology, 2015. [18] MARUTHUPANDI A, MUTHUPALANIAPPAN N, PANDIAN S R.Visual servoing of a 2-link underwater robot manipulator[C]//Underwater Technology, Chennai.DOI: 10.1109/UT.2015.7108226. [19] KEIGHOBADI J, FATEH M M, CHENARANI H.Adaptive fuzzy passivation control based on backstepping method for electrically driven robotic manipulators[C]// 2018 6th RSI International Conference on Robotics and Mechatronics (IcRoM).DOI: 10.1109/ICRoM.2018.8657516. [20] BURGER W, DEAN-LEON E, CHENG G.Robust second order sliding mode control for 6D position based visual servoing with a redundant mobile manipulator[C]// 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).DOI: 10.1109/HUMANOIDS.2015.7363494. [21] REZA R A,MOHAMMAD H K,AMIN N,et al.Robotic manipulator control based on an optimal fractional-order fuzzy PID approach:SiL real-time simulation[J].Soft Computing,2020,24(5): 3849- 3860. doi: 10.1007/s00500-019-04152-7 [22] 周孚成.基于图像的机器人视觉伺服控制[D].长沙: 中南大学, 2012.ZHOU Fucheng.Image-based robot visual servoing control[D].Changsha: Central South University, 2012. [23] 金玉成,高杨,刘继展,等.采摘机器人深度视觉伺服手−眼协调规划研究[J].农业机械学报,2021,52(6):18- 25,42. doi: 10.6041/j.issn.1000-1298.2021.06.002JIN Yucheng,GAO Yang,LIU Jizhan,et al.Hand-eye coordination planning with deep visual servo for harvesting robot[J].Transactions of the Chinese Society for Agricultural Machinery,2021,52(6):18- 25,42. doi: 10.6041/j.issn.1000-1298.2021.06.002 [24] 宋家慧,孔令美.采摘机器人视觉伺服策略研究:基于回归数据挖掘的[J].农机化研究,2016,38(12):81-85. doi: 10.3969/j.issn.1003-188X.2016.12.016SONG Jiahui,KONG Lingmei.Research on picking robot visual servo strategy:based on regression data mining[J].Journal of Agricultural Mechanization Research,2016,38(12):81-85. doi: 10.3969/j.issn.1003-188X.2016.12.016 [25] 刘广瑞,黄真,韩莉莉,等.基于Kalman滤波的图像雅克比矩阵在线估计[J].郑州大学学报(工学版),2013,34(1):95-98. doi: 10.3969/j.issn.1671-6833.2013.01.023LIU Guangrui,HUANG Zhen,HAN Lili,et al.Tracking of two dimensional based on Kalman filtering[J].Journal of Zhengzhou University (Engineering Science),2013,34(1):95-98. doi: 10.3969/j.issn.1671-6833.2013.01.023 [26] 王博,言勇华.基于图像的无标定视觉伺服系统[J].机械设计与研究,2019,35(3):135-139. doi: 10.13952/j.cnki.jofmdr.2019.0203WANG Bo,YAN Yonghua.Image based uncalibrated visual servoing system[J].Machine Design & Research,2019,35(3):135-139. doi: 10.13952/j.cnki.jofmdr.2019.0203 [27] 梁喜凤,彭明,路杰,等.基于自适应无迹卡尔曼滤波的采摘机械手视觉伺服控制方法[J].农业工程学报,2019,35(19):230-237. doi: 10.11975/j.issn.1002-6819.2019.19.028LIANG Xifeng,PENG Ming,LU Jie,et al.Servo control method of picking manipulator based on adaptive traceless Kalman filter[J].Transactions of the Chinese Society of Agricultural Engineering,2019,35(19):230-237. doi: 10.11975/j.issn.1002-6819.2019.19.028 [28] JIANG P,BAMFORTH L C A,FENG Z,et al.Indirect iterative learning control for a discrete visual servo without a camera-robot model[J].IEEE Transactions on Systems,Man,and Cybernetics:PartB,2007,37(4):863-876. doi: 10.1109/TSMCB.2007.895355 [29] 谷雨,李平,韩波.一种适用于局部特征的混合视觉伺服方法[J].控制与决策,2010,25(1):43-47. doi: 10.13195/j.cd.2010.01.46.guy.014GU Yu,LI Ping,HAN Bo.Hybrid visual servoing scheme applicable for local features[J].Control and Decision,2010,25(1):43-47. doi: 10.13195/j.cd.2010.01.46.guy.014 [30] 阳建峰.无标定视觉技术及其在机器人柔性检测系统中的应用与研究[D].长春: 长春工业大学, 2019.YANG Jianfeng.Uncalibrated vision technology and its application and research in robot flexible detection system[D].Changchun: Changchun University of Technology, 2019. [31] 宋健.基于图像的采摘机器人模糊视觉伺服系统研究[J].机床与液压,2010,38(10):65-67. doi: 10.3969/j.issn.1001-3881.2010.10.023SONG Jian.Research on image-based fuzzy visual servo for picking robot[J].Machine Tool & Hydraulics,2010,38(10):65-67. doi: 10.3969/j.issn.1001-3881.2010.10.023 [32] 赵庆波,赵德安,姬伟,等.采摘机器人视觉伺服控制系统设计[J].农业机械学报,2009,40(1):152-156.ZHAO Qingbo,ZHAO Dean,JI Wei,et al.Vision servoing control system of fruit harvesting robot[J].Transactions of the Chinese Society for Agricultural Machinery,2009,40(1):152-156. [33] MALIS,EZIO,CHAUMETTE,et al.2-1/2-D visual servoing[J].IEEE Transactions on Robotics & Automation,1999,15(2):238-250. [34] 李嘉惠,张丰收,崔浩阳.一种改进 RANSAC 算法的单应性矩阵估计方法[J].软件导刊,2020,19(2):149-152.LI Jiahui,ZHANG Fengshou,CUI Haoyang.A homography matrix estimation method based on improved RANSAC algorithm[J].Software Guide,2020,19(2):149-152. [35] 吴柔莞,徐智勇,张建林.基于无监督级联的亚像素单应矩阵估计[J].半导体光电,2022,43(1):158-163. doi: 10.16818/j.issn1001-5868.2021092601WU Rouwan,XU Zhiyong,ZHANG Jianlin.Sub-pixel homography matrix estimation based on unsupervised cascade[J].Semiconductor Optoelectronics,2022,43(1):158-163. doi: 10.16818/j.issn1001-5868.2021092601 [36] 徐同庆.基于机器视觉的苹果采摘机器人控制系统研究[D].镇江: 江苏科技大学, 2020.XU Tongqing.Research on control system of apple picking robot based on machine vision[D].Zhenjiang: Jiangsu University of Science and Technology, 2020.