Recent advances on vision-based robot learning by demonstration

Background: Traditionally, robots perform desired tasks with the aid of end-users analytically decomposing and manually programming. Robots controlled the program can only follow program instructions to move, which bring great difficulty to non-professional users. So it is meaningful to study new robot control paradigm for robots to accomplish tasks actively without the professional program. Vision-based robot learning by demonstration (vision-based LbD) is an effective approach that a robot can autonomously accomplish a task with help of the combination of the learning by demonstration (LbD) and vision sensing technology. Vision-based LbD allows robots to learn skills through ‘see-ing’ demonstrations of users with vision sensors. Vision-based LbD reduces the operation difficulty of robots and provides an intuitive manner for human interact with robot, especially for those users who have no professional program experience. Objective: Providing the references for researchers who work in related fields by reviewing recent advances of vision-based LbD. Methods: This paper reviews the latest patents and current representative articles related to vision-based LbD. The key methods of these references are introduced in the aspects of algorithms, innovations and principles. Results: The researches related to vision-based LbD in the last 5 years are classified, the advantages of different algorithms in these patents and articles are introduced and analyzed, the future developments and potential problems in this field are discussed. Conclusion: The main advantage of vision-based LbD is to allow users training robots for new tasks by the demonstration under the vision sensor without programming control. So, vision-based LbD provides an intuitive manner of robot learning by demonstration to solve the problem of human-robot interaction. Further improvement is required in the following aspects: Algorithm innovation, multiple demonstrations, many definitions of human action and so on. More patents on vision-based LbD should be in-vented.