Wearable Visuomotor Navigation and Mobility Assistance Device for the Partially Sighted and Visually Impaired in Unfamiliar Indoor Scenes

To improve the mobility and independence of the partially sighted and visually impaired (PSVI), a variety of assistive navigation devices with simultaneous localization and mapping technologies have been proposed. However, upon entering an unfamiliar indoor scene, PSVI users face the challenge of lacking dense prior maps for navigation. To well address this challenge, we propose an innovative wearable navigation assistance system for them in unfamiliar indoor scenes. To construct maps required for navigation, we generate topological semantic maps (TSM) from layout maps. To utilize TSM for localization, we propose to match the detected text nodes from signs with that of TSM for positioning. To significantly improve localization accuracy and mitigate instances of node mismatch between real-world scenes and TSM, we develop glasses equipped with binocular distance measurement, thereby enabling precise navigation toward signs for PSVI users. We determine the positions of obstacles around them through visual perception. In emergency situations requiring obstacle avoidance, our designed wristbands provide immediate and intuitive vibrational feedback to guide them. We deploy our algorithms in a computer that communicates with the users through multiple wearable devices. Finally, we conduct navigation experiments in a hospital. Experimental results show that our device can assist PSVI users in completing indoor navigation tasks safely and efficiently.