An Improved Image Stitching Method Based on Vision-Inertial Fusion

To enhance the accuracy of autonomous navigation in Unmanned Aerial Vehicles (UAVs) and boost their autonomous sensing and measuring abilities in challenging real-world flying scenarios. This paper focuses on the mapping process in autonomous navigation of UAVs. The work aims to solve the problems in traditional Oriented FAST and Rotated BRIEF (ORB) feature extraction algorithm where feature points are prone to accumulate in areas with rich texture, and the limitation of Oriented FAST and Rotated BRIEF - Simultaneous Localization and Mapping 3 (ORB-SLAM3) system to construct dense maps. An improved image stitching method of visual-inertial fusion for dense mapping is proposed based on ORB-SLAM3. and the proposed method uses the obtained panoramic images to create dense maps. Firstly, the Scale Invariant Feature Transform (SIFT) algorithm is utilized during feature point extraction. The Random Sample Consensus (RANSAC) algorithm is applied for feature point optimization to obtain the best feature points for matching. Secondly, combining the posture information obtained by IMU pre-integration and the visual information will be used to compensate for the errors caused by the speed of the UAV and different angles of aerial photography. Then eliminating redundant and repeated pixels, and stitching adjacent pictures will help to get a panoramic picture. Finally, generate a dense map from the sparse point cloud map generated by ORB-SLAM3. Experiments based on the AirSim are performed to validate the proposed method. The result shows that the SIFT feature extraction method outperforms other methods, the improved image stitching method can effectively eliminate ghosting and pixel information loss problems in various environments, and the dense maps can also be achieved.