Training musculoskeletal arm play taichi with deep reinforcement learning

Musculoskeletal robot arm driven by pneumatic artificial muscle actuators is secure, lightweight and compliant, which makes it an ideal solution for prosthetics and rehabilitation equipment. However, there exists conflicts between anatomical accuracy and engineering feasibility, and the nonlinearity of muscle actuators brings difficulty in accurate mathematical modeling. To overcome these problems, in this paper, we propose an optimized musculoskeletal arm design of ten muscles and four degrees-offreedom, employ Deep Deterministic Policy Gradient (DDPG) to train a data driven controller, and combine the off-policy reinforcement learning algorithm with Hindsight Experience Replay (HER) to deal with sparse rewards situation. TaiChi motion sequence is acquired using Perception Neuron and the trajectory tracking experiments are carried out in simulation platform. Experiment results demonstrate that, the controlled musculoskeletal arm is able to track the TaiChi trajectory and learn well in sparse rewards situation.