Research on Target Trajectory Planning Method of Humanoid Manipulators Based on Reinforcement Learning

The goal of most asymmetrically coordinated manipulative tasks of humanoid manipulators is multilevel. For example, a bottle cap screwing task is composed of several sub-objectives, such as reaching, grasping, aligning, and screwing. In addition, the flexible interaction requirements of dual-arm robots challenge the trajectory planning methods of manipulator with high dimensional and strong coupling characteristics. However, the traditional reinforcement learning algorithms cannot quickly learn and generate the required trajectories above. Based on the idea of multi-agent control, a dual-agent deep deterministic policy gradient algorithm is proposed in this paper, which uses two agents to simultaneously plan the coordinated trajectory of the left arm and the right arm online. This algorithm solves the problem of online trajectory planning for multi-objective tasks of humanoid manipulators. The design of observations and actions in the dual-agent structure can reduce the dimension and decouple the humanoid manipulators’ trajectory planning problem to a certain extent, thus speeding up the learning speed. Moreover, a reward function is constructed to realize the coordinated control between the two agents, to promote dual-agent to generate continuous trajectories for multi-objective tasks. Finally, the effectiveness of the proposed algorithm is verified in Baxter multi-objective task simulation environment under the Gym. The results show that this algorithm can quickly learn and online plan the coordinated trajectory of humanoid manipulators for multi-objective tasks.