Hydrogen Energy Storage System Participated Decentralized Voltage Control With Multi-Agent Deep Reinforcement Learning Method

With the development of power electronic technology, smart inverters and energy storage systems are progressively employed for voltage regulation in active distribution networks (ADNs). In this article, we incorporate hydrogen energy storage system (HESS) into distribution network voltage control and propose a cooperated voltage control framework. At first, we formulate a two-timescale voltage control problem considering the characteristics of different voltage regulation devices. HESS is accurately modeled and introduced into the fast timescale. To achieve a decentralized and efficient solution to this problem, we reformulate it as a two-timescale Markov games and then propose a modified multi-agent soft actor-critic (MASAC) algorithm to solve it. Specifically, the prioritized experience replay is introduced into MASAC algorithm, which is called PER-MASAC, to enhance the training process stability and improve the control performance. The proposed voltage control framework is tested with a modified IEEE 33-bus distribution system. The simulation results demonstrate that it can effectively mitigate the voltage fluctuation, reduce network loss and avoid the operational violation of HESS.