Safe Reinforcement Learning with Constraints: A Survey

Despite the significant achievements of reinforcement learning (RL) algorithms in multiple domains, their application in real-world scenarios still encounters numerous challenges. A primary concern is safety, which is also known as constraint satisfaction. The safety of agents needs to be ensured throughout the entire training process, even at every single time step. Therefore, the incorporation of safety constraints into RL needs to be considered. First, this paper summarizes three forms of constraints: soft, hard, and hybrid. Second, this paper elaborates on the specific implementation forms and applications of each type of constraint. In conclusion, the paper provides a comprehensive summary of these constraint forms.