Human hand motion analysis with multisensory information

In order to study and analyze human hand motions that contain multimodal information, a generalized framework integrating multiple sensors is proposed and consists of modules of sensor integration, signal preprocessing, correlation study of sensory information, and motion identification. Three types of sensors are integrated to simultaneously capture the finger angle trajectories, the hand contact forces, and the forearm electromyography (EMG) signals. To facilitate the rapid acquisition of human hand tasks, methods to automatically synchronize and segment manipulation primitives are developed in the signal preprocessing module. Correlations of the sensory information are studied by using Empirical Copula and demonstrate that there exist significant relationships between muscle signals and finger trajectories and between muscle signals and contact forces. In addition, recognizing different hand grasps and manipulations based on the EMG signals is investigated by using Fuzzy Gaussian Mixture Models (FGMMs) and results of comparative experiments show FGMMs outperform Gaussian Mixture Models and support vector machine with a higher recognition rate. The proposed framework integrating the state-of-the-art sensor technology with the developed algorithms provides researchers a versatile and adaptable platform for human hand motion analysis and has potential applications especially in robotic hand or prosthetic hand control and human-computer interaction.