Viiat-Hand: A Reach-and-Grasp Restoration System Integrating Voice Interaction, Computer Vision, Auditory and Tactile Feedback for Non-Sighted Amputees

For non-sighted and visually impaired (BVI) amputees, the combined loss of vision and grasping abilities turns the seemingly simple task of reaching and grasping into a significant challenge. This letter introduces a novel multi-sensory prosthesis system designed for BVI amputees to assist in perception, navigation, and grasping tasks. The system integrates voice interaction, environmental perception, grasp guidance, collaborative control, and auditory/tactile feedback. Specifically, it processes user commands, provides environmental data via auditory/tactile channels, and manages collaborative control of grasp gestures and wrist angles for stable object handling. The prototype, viiat-hand, was experimentally tested with eight non-disabled and four non-sighted subjects performing reach-and-grasp tasks, showing that users could accurately reach (average time:15.24 s) and securely grasp objects (average time:17.23 s) in an indoor setting. The system also proved to be user-friendly, requiring minimal training for users to become adept.