Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection

Huan Zhao; Gaoqi Liang; Guolong Liu; Junhua Zhao; Yan Xu

doi:10.1109/EI252483.2021.9713001

Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection

Huan Zhao^*
, Gaoqi Liang
, Guolong Liu
, Junhua Zhao
, Yan Xu

^*Corresponding author for this work

The Chinese University of Hong Kong, Shenzhen
Nanyang Technological University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Using deep reinforcement learning to control the wind farm is difficult to practically apply in the real environment due to high learning costs. Model-based reinforcement learning utilizes the data-driven model to simulate the environment and accelerate the learning speed. But the model needs a large amount of data to achieve an accurate result. Therefore, hybrid-model-based reinforcement learning is applied to address this problem by combining the knowledge-driven model with the model-based reinforcement learning framework. The Hybrid-Model-based Deep Deterministic Policy Gradient (HM-DDPG) algorithm with model selection method is proposed. Three scenarios in WFSim are built as the test environment. The simulation results show that the proposed method can accelerate the wind farm learning process and is more suitable for sizeable state-action space situations.

Original language	English
Title of host publication	5th IEEE Conference on Energy Internet and Energy System Integration
Subtitle of host publication	Energy Internet for Carbon Neutrality, EI2 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	750-755
Number of pages	6
ISBN (Electronic)	9781665434256
DOIs	https://doi.org/10.1109/EI252483.2021.9713001
State	Published - 2021
Externally published	Yes
Event	5th IEEE Conference on Energy Internet and Energy System Integration, EI2 2021 - Taiyuan, China Duration: 22 Oct 2021 → 25 Oct 2021

Publication series

Name	5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021

Conference

Conference	5th IEEE Conference on Energy Internet and Energy System Integration, EI2 2021
Country/Territory	China
City	Taiyuan
Period	22/10/21 → 25/10/21

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

Wind farm control
deep deterministic policy gradient
deep reinforcement learning
hybrid model

Access to Document

10.1109/EI252483.2021.9713001

Cite this

Zhao, H., Liang, G., Liu, G., Zhao, J., & Xu, Y. (2021). Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection. In 5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021 (pp. 750-755). (5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EI252483.2021.9713001

Zhao, Huan ; Liang, Gaoqi ; Liu, Guolong et al. / Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection. 5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 750-755 (5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021).

@inproceedings{f7b0144e734a4f0b9794d41a58b7666d,

title = "Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection",

abstract = "Using deep reinforcement learning to control the wind farm is difficult to practically apply in the real environment due to high learning costs. Model-based reinforcement learning utilizes the data-driven model to simulate the environment and accelerate the learning speed. But the model needs a large amount of data to achieve an accurate result. Therefore, hybrid-model-based reinforcement learning is applied to address this problem by combining the knowledge-driven model with the model-based reinforcement learning framework. The Hybrid-Model-based Deep Deterministic Policy Gradient (HM-DDPG) algorithm with model selection method is proposed. Three scenarios in WFSim are built as the test environment. The simulation results show that the proposed method can accelerate the wind farm learning process and is more suitable for sizeable state-action space situations.",

keywords = "Wind farm control, deep deterministic policy gradient, deep reinforcement learning, hybrid model",

author = "Huan Zhao and Gaoqi Liang and Guolong Liu and Junhua Zhao and Yan Xu",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE; 5th IEEE Conference on Energy Internet and Energy System Integration, EI2 2021 ; Conference date: 22-10-2021 Through 25-10-2021",

year = "2021",

doi = "10.1109/EI252483.2021.9713001",

language = "英语",

series = "5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "750--755",

booktitle = "5th IEEE Conference on Energy Internet and Energy System Integration",

address = "美国",

}

Zhao, H, Liang, G, Liu, G, Zhao, J & Xu, Y 2021, Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection. in 5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021. 5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021, Institute of Electrical and Electronics Engineers Inc., pp. 750-755, 5th IEEE Conference on Energy Internet and Energy System Integration, EI2 2021, Taiyuan, China, 22/10/21. https://doi.org/10.1109/EI252483.2021.9713001

Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection. / Zhao, Huan; Liang, Gaoqi; Liu, Guolong et al.
5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 750-755 (5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection

AU - Zhao, Huan

AU - Liang, Gaoqi

AU - Liu, Guolong

AU - Zhao, Junhua

AU - Xu, Yan

PY - 2021

Y1 - 2021

N2 - Using deep reinforcement learning to control the wind farm is difficult to practically apply in the real environment due to high learning costs. Model-based reinforcement learning utilizes the data-driven model to simulate the environment and accelerate the learning speed. But the model needs a large amount of data to achieve an accurate result. Therefore, hybrid-model-based reinforcement learning is applied to address this problem by combining the knowledge-driven model with the model-based reinforcement learning framework. The Hybrid-Model-based Deep Deterministic Policy Gradient (HM-DDPG) algorithm with model selection method is proposed. Three scenarios in WFSim are built as the test environment. The simulation results show that the proposed method can accelerate the wind farm learning process and is more suitable for sizeable state-action space situations.

AB - Using deep reinforcement learning to control the wind farm is difficult to practically apply in the real environment due to high learning costs. Model-based reinforcement learning utilizes the data-driven model to simulate the environment and accelerate the learning speed. But the model needs a large amount of data to achieve an accurate result. Therefore, hybrid-model-based reinforcement learning is applied to address this problem by combining the knowledge-driven model with the model-based reinforcement learning framework. The Hybrid-Model-based Deep Deterministic Policy Gradient (HM-DDPG) algorithm with model selection method is proposed. Three scenarios in WFSim are built as the test environment. The simulation results show that the proposed method can accelerate the wind farm learning process and is more suitable for sizeable state-action space situations.

KW - Wind farm control

KW - deep deterministic policy gradient

KW - deep reinforcement learning

KW - hybrid model

UR - https://www.scopus.com/pages/publications/85128203707

U2 - 10.1109/EI252483.2021.9713001

DO - 10.1109/EI252483.2021.9713001

M3 - 会议稿件

AN - SCOPUS:85128203707

T3 - 5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021

SP - 750

EP - 755

BT - 5th IEEE Conference on Energy Internet and Energy System Integration

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 5th IEEE Conference on Energy Internet and Energy System Integration, EI2 2021

Y2 - 22 October 2021 through 25 October 2021

ER -

Zhao H, Liang G, Liu G, Zhao J, Xu Y. Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection. In 5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 750-755. (5th IEEE Conference on Energy Internet and Energy System Integration: Energy Internet for Carbon Neutrality, EI2 2021). doi: 10.1109/EI252483.2021.9713001

Cooperative Wind Farm Control with Hybrid-Model-Based Deep Deterministic Policy Gradient and Model Selection

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this