Generative Adversarial Network with In-Betweening for Fine-Grained Skeleton-Based Action Generation

Human action generation has shown critical value in both industry application and academic research, especially fine-grained skeleton-based action generation. At present, there are several main ways to achieve fine-grained skeleton-based action generation. In this work, we utilize a generative adversarial network with in-betweening, in order to generate natural and continuous skeleton sequences and better capture complex motion details. In-betweening can be viewed as a prediction task based on the past and future context. Besides, it is convenient to control the result in a personalized way and combine the framework with other network structures to further improve the performance. In particular, two simple discriminators are applied on different timescales and LSTM serves as the foundation of the action generator because of its capability of effectively capturing long-term dependencies and processing sequence data of different lengths. The target noise added to the input of the generator can contribute to the robustness of the system and further improve the performance. Moreover, a novel Tai Chi dataset composed of fine-grained skeleton-based action data is created using high quality motion capture technology. The model is evaluated on two fine-grained skeleton-based datasets, LaFAN1 and Tai Chi, which contain common daily actions and professional sports actions respectively, and it achieves superior performance both qualitatively and quantitatively.