Self-Supervised Learning of Camera Ego-Motion From Optical Flow

Estimating the relative pose between two consecutive frames is essential in robotics and computer vision. Recent research has shown that such ego-motion estimators can be learned from unlabeled monocular videos by joint learning of a pose network and a depth network, which eliminates the need for ground-truth data. However, the training speed is slow, the training memory is large, and the accuracy of pose prediction is not satisfied. In this article, we propose a novel and efficient self-supervised learning method to predict the camera ego-motion. In light of the uniqueness of flow decomposition, we prove two innovative losses based on optical flow, that is, epipolar loss and flow decomposition loss, are proposed. The proposed method does not require an additional depth network, which leads to a faster training process and less training memory. Moreover, to overcome the limitation of performance brought by the presence of moving objects during training, we design a confidence network to automatically localize moving objects and mask the corresponding regions. Experiments across various public datasets and self-collected datasets demonstrate that our ego-motion network achieves state-of-the-art accuracy over long video sequences. Furthermore, our method not only outperforms other self-supervised approaches in terms of accuracy but also surpasses some supervised approaches.