Efficient Optimization Control Strategy for Air Source Heat Pump: A perspective from MPC-guided Deep Reinforcement Learning

Air Source Heat Pumps (ASHPs) are highly efficient heating systems whose performance strongly depends on intelligent control strategies. Traditional rule-based controls often yield suboptimal efficiency, while advanced methods like Model Predictive Control (MPC) can save significant energy but require accurate models. Recently, Deep Reinforcement Learning (DRL) has emerged as a model-free alternative capable of achieving MPC-like performance in building heating control. However, pure DRL can suffer from long training times and stability issues, whereas MPC alone may struggle with model mismatches and limited prediction horizons. In this paper, we propose an MPC-guided DRL control strategy for ASHPs that synergistically combines the strengths of both methods. The approach uses MPC to guide the DRL agent’s exploration and policy learning, leveraging MPC’s short-term optimal decisions to improve DRL training stability and safety. The MPC-guided DRL controller consistently outperforms baseline rule-based control and standalone MPC or DRL. It achieves higher energy while maintaining indoor comfort within desired ranges, significantly reducing peak demand and runtime variability. These results demonstrate that MPC-guided DRL is a promising and efficient optimization control strategy for heat pumps, combining data-driven adaptability with model-based foresight.