Fuse only what matters: Degeneracy-aware multi-sensor fusion for LiDAR-Inertial-Visual SLAM

Recent advances in LiDAR-Inertial-Visual SLAM typically adopt an “all-in” fusion strategy, indiscriminately integrating all sensor inputs regardless of necessity. While this approach aims to enhance robustness, it often incurs excessive computational overhead and can degrade performance by introducing noise from lower-precision modalities in non-degenerate scenarios. To address these limitations, we propose a degeneracy-aware framework anchored by a Selective Kalman Filter that fundamentally resolves “when” and “how” to fuse multi-modal data. Specifically, our approach introduces a rigorous detection metric that evaluates the coupled geometric constraints within the state space, enabling the filter to distinguish between actual system failure and benign environmental features. By accurately identifying the precise timing and direction of sensor degeneracy, our system selectively incorporates visual measurements on an on-demand basis. This mechanism integrates visual constraints exclusively into the dimensions where the primary LiDAR-inertial subsystem fails. Such a selective approach effectively eliminates redundant computation and prevents the contamination of high-precision states by less reliable visual data during nominal operation. Our extensive experimental validation spans a wide variety of challenging scenarios, including texture-less indoor corridors and large-scale open environments, verifying the system’s ability to maintain high localization accuracy where single-modality approaches typically fail. Extensive evaluations demonstrate that our method achieves superior localization accuracy and robustness while significantly enhancing computational efficiency compared to state-of-the-art “all-in” frameworks.