Adaptive self-correction network for human motion prediction

Human motion prediction aims to generate future poses from the observed historical human motion sequence. It is fundamental to many intelligent systems, e.g., human–robot interaction and self-driving. Though the existing encoder–decoder methods obtain good performance in some scenarios, there is still a gap between their prediction results and ground truth in many cases. In this work, we propose to estimate the deviation between the decoding results (which will be referred to as the conventional human motion prediction) and the groundtruth, and integrate this estimation with the conventional prediction results to derive the corrected human motion prediction. In this way, our method can self-correct the conventional prediction results based on a preliminary estimated deviation from it to the groundtruth, and thus enhance the performance. In our work, we adopt five independent lightweight branches rather than a global estimator to estimate the deviation of the five human body components (i.e., left arm, right arm, torso, left leg, and right leg). Based on this component-wise deviation estimation strategy, we propose a Fixed Self-Correction Network (FSCNet) for human motion prediction to obtain enhanced performance. Recognizing that not all joints exhibit the same motion dynamics inside one given body component, we further propose the Adaptive Self-Correction Network (ASCNet) to let these five estimators adaptively capture the correlated deviations and thus enhance human motion prediction performance. Extensive experiments on three large datasets (Human3.6M, CMU-Mocap, and 3DPW) validate the superiority of our proposed FSCNet and ASCNet over the established works.