Artificial Intelligence in Robotic Manipulators: Exploring Object Detection and Grasping Innovations

Montassar Aidi Sharif; Hanan  Hameed Ismael; Muamar  Almani Jasim; Farah  Zuhair Jasim

doi:10.30572/2018/KJE/160109

Authors

Montassar Aidi Sharif Electronic and Control Engineering Department, Technical Engineering College –Kirkuk, Northern Technical University, Iraq https://orcid.org/0000-0002-9879-0631
Hanan Hameed Ismael Electronic and Control Engineering Department, Technical Engineering College –Kirkuk, Northern Technical University, Iraq
Muamar Almani Jasim Computer Engineering Department, Technical Engineering College –Kirkuk, Northern Technical University, Iraq
Farah Zuhair Jasim Electronic and Control Engineering Department, Technical Engineering College –Kirkuk, Northern Technical University, Iraq

DOI:

https://doi.org/10.30572/2018/KJE/160109

Keywords:

Robotics manipulator, Object detection, Robot Grasping, Artificial Intelligence, YOLO

Abstract

The importance of deep learning has heralded transforming changes across different technological domains, not least in the enhancement of robotic arm functionalities of object detection’s and grasping. This paper is aimed to review recent and past studies to give a comprehensive insight to focus in exploring cutting-edge deep learning methodologies to surmount the persistent challenges of object detection and precise manipulation by robotic arms. By integrating the iterations of the You Only Look Once (YOLO) algorithm with deep learning models, our study not only advances the innovations in robotic perception but also significantly improves the accuracy of robotic grasping in dynamic environments. Through a comprehensive exploration of various deep learning techniques, we introduce many approaches that enable robotic arms to identify and grasp objects with unprecedented precision, thereby bridging a critical gap in robotic automation. Our findings demonstrate a marked enhancement in the robotic arm’s ability to adapt to and interact with its surroundings, opening new avenues for automation in industrial, medical, and domestic applications. The impact of this research extends lays the groundwork for future developments in robotic autonomy, offering insights into the integration of deep learning algorithms with robotic systems. This also serves as a beacon for future research aimed at fully unleashing the potential of robots as autonomous agents in complex, real-world settings.

Downloads

Download data is not yet available.

References

Advincula, A.P. and Wang, K., 2009. Evolving role and current state of robotics in minimally invasive gynecologic surgery. Journal of Minimally Invasive Gynecology, 16(3), pp.291-301. DOI: https://doi.org/10.1016/j.jmig.2009.03.003

Arulkumaran, K. et al., 2017. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34(6), pp.26-38. DOI: https://doi.org/10.1109/MSP.2017.2743240

Bochkovskiy, A., Wang, C.Y. and Liao, H.Y.M., 2020. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.

Chen, C.H., Huang, H.P. and Lo, S.Y., 2011. Stereo-based 3D localization for grasping known objects with a robotic arm system. In 2011 9th World Congress on Intelligent Control and Automation. IEEE.

Chen, G.H., Jun-Yi, W. and Ai-Jun, Z., 2019. Transparent object detection and location based on RGB-D camera. Journal of Physics: Conference Series, 1183(1). DOI: https://doi.org/10.1088/1742-6596/1183/1/012011

Chen, L. et al., 2023. Perceiving unseen 3D objects by poking the objects. arXiv preprint arXiv:2302.13375. DOI: https://doi.org/10.1109/ICRA48891.2023.10160338

Chen, Q. et al., 2022. Vision-based impedance control of a 7-DOF robotic manipulator for pick-and-place tasks in grasping fruits.

Chen, Y.L., Cai, Y.R. and Cheng, M.Y., 2023. Vision-based robotic object grasping: a deep reinforcement learning approach. Machines, 11(2), p.275. DOI: https://doi.org/10.3390/machines11020275

Chen, Z. et al., 2022. Towards generalization and data efficient learning of deep robotic grasping. In 2022 IEEE 17th Conference on Industrial Electronics and Applications (ICIEA). IEEE. DOI: https://doi.org/10.1109/ICIEA54703.2022.10006045

Choi, C. et al., 2018. Learning object grasping for soft robot hands. IEEE Robotics and Automation Letters, 3(3), pp.2370-2377. DOI: https://doi.org/10.1109/LRA.2018.2810544

Cong, X. et al., 2023. A review of YOLO object detection algorithms based on deep learning. Frontiers in Computing and Intelligent Systems, 4(2), pp.17-20. DOI: https://doi.org/10.54097/fcis.v4i2.9730

Czajewski, W. and Kołomyjec, K., 2017. 3D object detection and recognition for robotic grasping based on RGB-D images and global features. Foundations of Computing and Decision Sciences, 42(3), pp.219-237. DOI: https://doi.org/10.1515/fcds-2017-0011

Dafoe, A., Bachrach, Y., Hadfield, G., Horvitz, E., Larson, K. and Graepel, T., 2021. Cooperative AI: machines must learn to find common ground. DOI: https://doi.org/10.1038/d41586-021-01170-0

Daugherty, P.R. and Wilson, H.J., 2018. Human+ machine: Reimagining work in the age of AI. Harvard Business Press.

Dong, H. et al., 2020. Deep Reinforcement Learning. Springer. DOI: https://doi.org/10.1007/978-981-15-4095-0

Du, G. et al., 2021. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artificial Intelligence Review, 54(3), pp.1677-1734. DOI: https://doi.org/10.1007/s10462-020-09888-5

Ekvall, S., Kragic, D. and Hoffmann, F., 2005. Object recognition and pose estimation using color cooccurrence histograms and geometric modeling. Image and Vision Computing, 23(11), pp.943-955. DOI: https://doi.org/10.1016/j.imavis.2005.05.006

Fairag, M., Almahdi, R.H., Siddiqi, A.A., Alharthi, F.K., Alqurashi, B.S., Alzahrani, N.G., Alsulami, A., Alshehri, R., Alzahrani, N.G., Alsulami, A.S. et al., 2024. Robotic revolution in surgery: Diverse applications across specialties and future prospects review article. Cureus, 16(1). DOI: https://doi.org/10.7759/cureus.52148

Fan, Q., Rao, Q. and Huang, H., 2023. Multitarget flexible grasping detection method for robots in unstructured environments. CMES-Computer Modeling in Engineering & Sciences, 137(2). DOI: https://doi.org/10.32604/cmes.2023.028369

Fullan, M. and Langworthy, M., 2013. Towards a new end: New pedagogies for deep learning.

Gai, R., Chen, N. and Yuan, H., 2023. A detection algorithm for cherry fruits based on the improved YOLO-v4 model. Neural Computing and Applications, 35(19), pp.13895-13906. DOI: https://doi.org/10.1007/s00521-021-06029-z

Gao, M. et al., 2021. A hybrid YOLOv4 and particle filter based robotic arm grabbing system in nonlinear and non-Gaussian environment. Electronics, 10(10), p.1140. DOI: https://doi.org/10.3390/electronics10101140

Huang, Y.Q. et al., 2020. Optimized YOLOv3 algorithm and its application in traffic flow detections. Applied Sciences, 10(9), p.3079. DOI: https://doi.org/10.3390/app10093079

Hussain, M., 2023. When, where, and which? Navigating the intersection of computer vision and generative AI for strategic business integration. IEEE Access, 11, pp.127202-127215. DOI: https://doi.org/10.1109/ACCESS.2023.3332468

Issa, Abbas H., and Ali H. Majeed. “Intelligent Sensor Fault Detection Based on Soft Computing”. Kufa Journal of Engineering, vol. 4, no. 1, Jan. 2014, pp. 113-24, doi:10.30572/2018/KJE/411246. DOI: https://doi.org/10.30572/2018/KJE/411246

Jain, S.K. et al., 2023. Articulated robot arm for garbage disposal in hospital environment. In ITM Web of Conferences, 56. EDP Sciences. DOI: https://doi.org/10.1051/itmconf/20235601002

Jiang, P. et al., 2020. Depth image-based deep learning of grasp planning for textureless planar-faced objects in vision-guided robotic bin-picking. Sensors, 20(3), p.706. DOI: https://doi.org/10.3390/s20030706

Kang, H., Zhou, H. and Chen, C., 2020. Visual perception and modeling for autonomous apple harvesting. IEEE Access, 8, pp.62151-62163. DOI: https://doi.org/10.1109/ACCESS.2020.2984556

Kasaei, H. and Kasaei, M., 2023. MVGrasp: Real-time multi-view 3D object grasping in highly cluttered environments. Robotics and Autonomous Systems, 160, p.104313. DOI: https://doi.org/10.1016/j.robot.2022.104313

Kasaei, H. et al., 2021. Simultaneous multi-view object detection and grasping in open-ended domains. arXiv preprint arXiv:2106.01866.

Kaymak, C. and Aysegul, U.C.A.R., 2018. Implementation of object detection and detection algorithms on a robotic arm platform using Raspberry Pi. In 2018 International Conference on Artificial Intelligence and Data Processing (IDAP). IEEE. DOI: https://doi.org/10.1109/IDAP.2018.8620916

Kheder, Harem Ali. “HUMAN-COMPUTER INTERACTION: ENHANCING USER EXPERIENCE IN INTERACTIVE SYSTEMS”. Kufa Journal of Engineering, vol. 14, no. 4, Oct. 2023, pp. 23-41, doi:10.30572/2018/KJE/140403. DOI: https://doi.org/10.30572/2018/KJE/140403

Kim, J.H. et al., 2022. Object detection and classification based on YOLO-v5 with improved maritime dataset. Journal of Marine Science and Engineering, 10(3), p.377. DOI: https://doi.org/10.3390/jmse10030377

Koşer, H.E., 2023. Determination of angular status and dimensional properties of objects for grasping with robot arm. IEEE Latin America Transactions, 21(2), pp.335-343. DOI: https://doi.org/10.1109/TLA.2023.10015227

Kragic, D. and Christensen, H.I., 2002. Model based techniques for robotic servoing and grasping. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 1. IEEE. DOI: https://doi.org/10.1109/IRDS.2002.1041405

Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J. and Quillen, D., 2018. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research, 37(4-5), pp.421-436. DOI: https://doi.org/10.1177/0278364917710318

Lin, C.C. et al., 2016. Vision based object grasping of industrial manipulator. In 2016 International Conference on Advanced Robotics and Intelligent Systems (ARIS). IEEE. DOI: https://doi.org/10.1109/ARIS.2016.7886613

Liu, J. et al., 2023. Design of a virtual multi-interaction operation system for hand-eye coordination of grape harvesting robots. Agronomy, 13(3), p.829. DOI: https://doi.org/10.3390/agronomy13030829

Liu, N. et al., 2022. Collaborative viewpoint adjusting and grasping via deep reinforcement learning in clutter scenes. Machines, 10(12), p.1135. DOI: https://doi.org/10.3390/machines10121135

Mao, Q.C. et al., 2019. Mini-YOLOv3: Real-time object detector for embedded applications. IEEE Access, 7, pp.133529-133538. DOI: https://doi.org/10.1109/ACCESS.2019.2941547

Mohammed, M.M., Al-Khafaji, M.M. and Abbas, T.F., 2023. Smart robot vision for a pick and place robotic system. Engineering and Technology Journal, 41(6), pp.1-15. DOI: https://doi.org/10.30684/etj.2023.135966.1292

Moran, M.E., 2007. Evolution of robotic arms. Journal of Robotic Surgery, 1(2), pp.103-111. DOI: https://doi.org/10.1007/s11701-006-0002-x

Prattichizzo, D., Pozzi, M., Baldi, T.L., Malvezzi, M., Hussain, I., Rossi, S. and Salvietti, G., 2021. Human augmentation by wearable supernumerary robotic limbs: review and perspectives. Progress in Biomedical Engineering, 3(4), p.042005. DOI: https://doi.org/10.1088/2516-1091/ac2294

Qi, H. and Gong, S., 2023. Network architecture based on improved dense-fusion algorithm research on the detection and grasping method of robotic arm. In International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 12645. SPIE. DOI: https://doi.org/10.1117/12.2681054

Rakhimkul, S. et al., 2019. Autonomous object detection and grasping using deep learning for design of an intelligent assistive robot manipulation system. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE. DOI: https://doi.org/10.1109/SMC.2019.8914465

Redmon, J. and Farhadi, A., 2018. YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767.

Ren, Y. et al., 2018. Vision based object grasping of robotic manipulator. In 24th International Conference on Automation and Computing (ICAC). IEEE. DOI: https://doi.org/10.23919/IConAC.2018.8749001

Sekkat, H. et al., 2021. Vision-based robotic arm control algorithm using deep reinforcement learning for autonomous objects grasping. Applied Sciences, 11(17), p.7917. DOI: https://doi.org/10.3390/app11177917

Shahria, M.T. et al., 2022. A comprehensive review of vision-based robotic applications: Current state, components, approaches, barriers, and potential solutions. Robotics, 11(6), p.139. DOI: https://doi.org/10.3390/robotics11060139

She, Q. et al., 2020. Openloris-object: A robotic vision dataset and benchmark for lifelong deep learning. In 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE. DOI: https://doi.org/10.1109/ICRA40945.2020.9196887

Sun, T. et al., 2023. A detection method for soft objects based on the fusion of vision and haptics. Biomimetics, 8(1), p.86. DOI: https://doi.org/10.3390/biomimetics8010086

Tutsoy, O., 2023. A review of recent advancements in deep machine learning, artificial intelligence, object detection, and human-robot interactions approaches for assistive robotics. Ph.D., Fatma Gongor.

Wang, Q. et al., 2023. Design, integration, and evaluation of a robotic peach packaging system based on deep learning. Computers and Electronics in Agriculture, 211, p.108013. DOI: https://doi.org/10.1016/j.compag.2023.108013

Wei, A.H. and Chen, B.Y., 2020. Robotic object recognition and grasping with a natural background. International Journal of Advanced Robotic Systems, 17(2), p.1729881420921102. DOI: https://doi.org/10.1177/1729881420921102

Xiong, S., Tziafas, G. and Kasaei, H., 2023. Enhancing fine-grained 3D object detection using hybrid multi-modal vision transformer-CNN models. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023). DOI: https://doi.org/10.1109/IROS55552.2023.10342235

Yu, J. and Zhang, W., 2021. Face mask wearing detection algorithm based on improved YOLO-v4. Sensors, 21(9), p.3263. DOI: https://doi.org/10.3390/s21093263

Yu, Y. et al., 2020. Real-time visual localization of the picking points for a ridge-planting strawberry harvesting robot. IEEE Access, 8, pp.116556-116568. DOI: https://doi.org/10.1109/ACCESS.2020.3003034

Zarif, M.I.I. et al., 2022. A vision-based object detection and localization system in 3D environment for assistive robots’ manipulation. In Proceedings of the 9th International Conference of Control Systems, and Robotics (CDSR’22). DOI: https://doi.org/10.11159/cdsr22.112

Zhao, Y., Shi, Y. and Wang, Z., 2022. The improved YOLOv5 algorithm and its application in small target detection. In International Conference on Intelligent Robotics and Applications. Springer, pp.679-688. DOI: https://doi.org/10.1007/978-3-031-13841-6_61

Zhong, M. et al., 2019. Assistive grasping based on laser-point detection with application to wheelchair-mounted robotic arms. Sensors, 19(2), p.303. DOI: https://doi.org/10.3390/s19020303