Visual guidance method for artificial assembly in visual blind areas based on augmented reality

Manual assembly in blind areas is impeded by unknown assembly area paths and the assembler’s obstructed vision, which greatly affects assembly efficiency and accuracy. To solve this type of difficult blind area assembly problem, a visual guidance method for blind area manual assembly using augmented reality assistance combined with path planning and pose solving has been proposed. In the proposed framework, a Denavit-Hartenberg (D-H) parameter model of the human arm and the arm shape angle was first constructed to solve the forward and inverse kinematics, and the changes in joint angles during motion were obtained. Using the designed Rapidly exploring Random Tree*-Artificial Potential Field (RRT*-APF) algorithm, feasible collision-free arm movement paths were then searched in the assembly area, and the path points and scene models containing assembly position information were entered into augmented reality devices for visualization. At the same time, to obtain the real-time status of the arm end during blind area assembly, the Inertial Measurement Unit (IMU) fusion-extended Kalman filtering algorithm was used to analyze and solve the spatial geometric pose of the simplified rigid link model of the arm, thus achieving assisted positioning of the end of the arm in blind area scenarios. Finally, a comprehensive experiment was conducted by constructing a blind area assembly test scenario. Experimental results showed that this method can significantly improve the efficiency of blind area assembly operations and effectively reduce assembly error rates.