An effective framework with hybrid augmentation for visual reinforcement learning generalization

Current studies in Visual Reinforcement Learning focus on developing policies to acclimate to unknown environments through data augmentation. This paper aims to develop a new methodology to improve upon existing results. To this end, we first categorize existing methods into three groups based on the focus of augmentation: Task-Aware Augmentation, Image-Processing Augmentation, and Image-Scene Augmentation. Subsequently, we establish a unified framework that integrates these three augmentation categories. The core of our framework is hybrid data augmentation, which enhances data diversity. In this framework, we employ hyperspherical space and regularization techniques to address the side effects of such augmentation, specifically the discrepancy between augmented and original data, as well as the instability associated with hybrid augmentation. Finally, we evaluate the proposed framework across three benchmarks, demonstrating its significant advantages over current state-of-the-art methods. Notably, our framework outperforms existing approaches by an average of 4.59 % across 10 tasks in DMC-GB, 28.81 % across 6 tasks in Robosuite, and 20.50 % across 4 tasks in Adroit. The code for our framework will be released at https://github.com/csufangyu/MuHA.