Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems

Yucheng Chen; Yupeng Zhu; Shaohai Wang; Liyuan Yin; Hongming Zhu; Xingjian Sun; Chengwei Wu

doi:10.1109/CAC59555.2023.10451252

Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems

Yucheng Chen
, Yupeng Zhu
, Shaohai Wang
, Liyuan Yin
, Hongming Zhu
, Xingjian Sun
, Chengwei Wu

School of Astronautics

Harbin Institute of Technology
Tianjin Zero-One Intelligent Technology Co., Ltd
Nantong University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper investigates the robust adaptive optimal control problem for linear systems in the presence of matching uncertainties. A nominal controller utilizing Q-Iearning algorithm is designed for the nominal linear system without uncertainties. Introducing a disturbance observer serves the purpose of actively estimating uncertainties, enabling proactive compensation for matching uncertainties. The analysis of the closed-loop system's stability under the robust adaptive optimal controller is conducted, presenting sufficient conditions to ensure its stability. Finally, the control algorithm is validated using a two-wheeled mobile robot as the experimental platform.

Original language	English
Title of host publication	Proceedings - 2023 China Automation Congress, CAC 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1520-1525
Number of pages	6
ISBN (Electronic)	9798350303759
DOIs	https://doi.org/10.1109/CAC59555.2023.10451252
State	Published - 2023
Event	2023 China Automation Congress, CAC 2023 - Chongqing, China Duration: 17 Nov 2023 → 19 Nov 2023

Publication series

Name	Proceedings - 2023 China Automation Congress, CAC 2023

Conference

Conference	2023 China Automation Congress, CAC 2023
Country/Territory	China
City	Chongqing
Period	17/11/23 → 19/11/23

Keywords

Disturbance observer
Reinforcement learning
Robust optimal control

Access to Document

10.1109/CAC59555.2023.10451252

Cite this

Chen, Y., Zhu, Y., Wang, S., Yin, L., Zhu, H., Sun, X., & Wu, C. (2023). Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems. In Proceedings - 2023 China Automation Congress, CAC 2023 (pp. 1520-1525). (Proceedings - 2023 China Automation Congress, CAC 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CAC59555.2023.10451252

@inproceedings{673e6e74e5ac4dd78919f9193a04e051,

title = "Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems",

abstract = "This paper investigates the robust adaptive optimal control problem for linear systems in the presence of matching uncertainties. A nominal controller utilizing Q-Iearning algorithm is designed for the nominal linear system without uncertainties. Introducing a disturbance observer serves the purpose of actively estimating uncertainties, enabling proactive compensation for matching uncertainties. The analysis of the closed-loop system's stability under the robust adaptive optimal controller is conducted, presenting sufficient conditions to ensure its stability. Finally, the control algorithm is validated using a two-wheeled mobile robot as the experimental platform.",

keywords = "Disturbance observer, Reinforcement learning, Robust optimal control",

author = "Yucheng Chen and Yupeng Zhu and Shaohai Wang and Liyuan Yin and Hongming Zhu and Xingjian Sun and Chengwei Wu",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 China Automation Congress, CAC 2023 ; Conference date: 17-11-2023 Through 19-11-2023",

year = "2023",

doi = "10.1109/CAC59555.2023.10451252",

language = "英语",

series = "Proceedings - 2023 China Automation Congress, CAC 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1520--1525",

booktitle = "Proceedings - 2023 China Automation Congress, CAC 2023",

address = "美国",

}

Chen, Y, Zhu, Y, Wang, S, Yin, L, Zhu, H, Sun, X & Wu, C 2023, Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems. in Proceedings - 2023 China Automation Congress, CAC 2023. Proceedings - 2023 China Automation Congress, CAC 2023, Institute of Electrical and Electronics Engineers Inc., pp. 1520-1525, 2023 China Automation Congress, CAC 2023, Chongqing, China, 17/11/23. https://doi.org/10.1109/CAC59555.2023.10451252

Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems. / Chen, Yucheng; Zhu, Yupeng; Wang, Shaohai et al.
Proceedings - 2023 China Automation Congress, CAC 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 1520-1525 (Proceedings - 2023 China Automation Congress, CAC 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems

AU - Chen, Yucheng

AU - Zhu, Yupeng

AU - Wang, Shaohai

AU - Yin, Liyuan

AU - Zhu, Hongming

AU - Sun, Xingjian

AU - Wu, Chengwei

PY - 2023

Y1 - 2023

N2 - This paper investigates the robust adaptive optimal control problem for linear systems in the presence of matching uncertainties. A nominal controller utilizing Q-Iearning algorithm is designed for the nominal linear system without uncertainties. Introducing a disturbance observer serves the purpose of actively estimating uncertainties, enabling proactive compensation for matching uncertainties. The analysis of the closed-loop system's stability under the robust adaptive optimal controller is conducted, presenting sufficient conditions to ensure its stability. Finally, the control algorithm is validated using a two-wheeled mobile robot as the experimental platform.

AB - This paper investigates the robust adaptive optimal control problem for linear systems in the presence of matching uncertainties. A nominal controller utilizing Q-Iearning algorithm is designed for the nominal linear system without uncertainties. Introducing a disturbance observer serves the purpose of actively estimating uncertainties, enabling proactive compensation for matching uncertainties. The analysis of the closed-loop system's stability under the robust adaptive optimal controller is conducted, presenting sufficient conditions to ensure its stability. Finally, the control algorithm is validated using a two-wheeled mobile robot as the experimental platform.

KW - Disturbance observer

KW - Reinforcement learning

KW - Robust optimal control

UR - https://www.scopus.com/pages/publications/85189295075

U2 - 10.1109/CAC59555.2023.10451252

DO - 10.1109/CAC59555.2023.10451252

M3 - 会议稿件

AN - SCOPUS:85189295075

T3 - Proceedings - 2023 China Automation Congress, CAC 2023

SP - 1520

EP - 1525

BT - Proceedings - 2023 China Automation Congress, CAC 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 China Automation Congress, CAC 2023

Y2 - 17 November 2023 through 19 November 2023

ER -

Chen Y, Zhu Y, Wang S, Yin L, Zhu H, Sun X et al. Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems. In Proceedings - 2023 China Automation Congress, CAC 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 1520-1525. (Proceedings - 2023 China Automation Congress, CAC 2023). doi: 10.1109/CAC59555.2023.10451252

Reinforcement Learning and Disturbance Observer Based Optimal Control for Uncertain Systems

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this