Vision-based Task Learning and Manipulation for Humanoid Muscle-skeleton Robotic Arm

Humanoid muscle-skeleton robots have always been a hot topic in the field of robotics research due to their unique human-like body structure. People also expect it to complete some tasks as human beings, such as operating equipment or grasping objects. However, enabling humanoid robots to learn and perform various tasks is still difficult due to the complexity of integrating perception, decision making, and control. In this paper, we proposed vision-based task learning and manipulation for a humanoid muscle-skeleton robotic arm. This robotic arm consists of four McKibben muscles and can imitate the movement of human arms. For each new task, we first assist the robotic arm in completing the task and use a depth camera to collect data in progress, including the depth image and the color image. Then these data are used to train the motion prediction network. When the robotic arm performs a learned task, the motion prediction network can predict the next action to be taken (i.e., air pressure of McKibben muscles) and finally complete the task. The experimental results show that the robotic arm can quickly learn a new task and complete it with a high success rate.