Deep Reinforcement Learning-Assisted Convex Programming for AC Unit Commitment and Its Variants

Unit commitment (UC) is one of the core decisions in electricity markets and system planning. Existing optimization methods either use DC approximation of network constraints or decompose and relax them into subproblems, which may lead to suboptimal/infeasible decisions or require execution times longer than those for the industry. Recent advances in machine learning techniques have proven their ability to find fast near-optimal UC decisions. However, they suffer from the curses of dimensionality and/or lack feasibility guarantees. This paper proposes a new hybrid deep reinforcement learning algorithm for the UC problem with AC network constraints (AC-UC). The problem is formulated as a modified-action Markov decision process model and the fundamental soft actor-critic (SAC) algorithm is enhanced to deal with hybrid action spaces. To ensure the solution feasibility, two action filters are designed based on logic constraints of generation units and sequential convex programming for the discrete and continuous actions, respectively. The hybrid SAC is improved by embedding the two filters and a prioritized experience buffer for fast convergence and stable learning. Simulation results on four test systems verify the effectiveness of the proposed approach and the superiority over traditional methods in computational efficiency and feasibility. Moreover, the presented method is extended to solve stochastic UC and security-constraint UC problems.