Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking

Aohua Liu; Bo Zhang; Weiliang Chen; Yiang Luo; Shuxian Fang; Ouyang Zhang; Zhuang Liu; Zhenhuan Wang; Jianxing Liu

doi:10.1109/CAC57257.2022.10055583

Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking

Aohua Liu
, Bo Zhang
, Weiliang Chen^*
, Yiang Luo
, Shuxian Fang
, Ouyang Zhang
, Zhuang Liu
, Zhenhuan Wang
, Jianxing Liu

^*Corresponding author for this work

School of Astronautics

Harbin Institute of Technology
Suihua Power Supply Company State Grid

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

This paper investigates the trajectory tracking control method for a robotic manipulator with uncertainties. A compound controller combining a traditional control law with deep reinforcement learning is developed to improve tracking accuracy and adaptability. The model-based control method is able to increase sampling efficiency for learning control strategy. The introduction of deep reinforcement learning based on soft actor-critic structure and Lyapunov function constrain enables the system to compensate unknown uncertainties and remain stable. Eventually, a 3-DOF manipulator is used to show the effectiveness of the proposed controller. Comparative simulation results demonstrate that the compound controller acquires higher tracking accuracy than the pure model-based control method.

Original language	English
Title of host publication	Proceedings - 2022 Chinese Automation Congress, CAC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2740-2745
Number of pages	6
ISBN (Electronic)	9781665465335
DOIs	https://doi.org/10.1109/CAC57257.2022.10055583
State	Published - 2022
Event	2022 Chinese Automation Congress, CAC 2022 - Xiamen, China Duration: 25 Nov 2022 → 27 Nov 2022

Publication series

Name	Proceedings - 2022 Chinese Automation Congress, CAC 2022
Volume	2022-January

Conference

Conference	2022 Chinese Automation Congress, CAC 2022
Country/Territory	China
City	Xiamen
Period	25/11/22 → 27/11/22

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

3-DOF
Deep reinforcement learning
trajectory tracking control
uncertain robotic manipulators

Access to Document

10.1109/CAC57257.2022.10055583

Cite this

Liu, A., Zhang, B., Chen, W., Luo, Y., Fang, S., Zhang, O., Liu, Z., Wang, Z., & Liu, J. (2022). Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking. In Proceedings - 2022 Chinese Automation Congress, CAC 2022 (pp. 2740-2745). (Proceedings - 2022 Chinese Automation Congress, CAC 2022; Vol. 2022-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CAC57257.2022.10055583

@inproceedings{78bce00ef10644a1bde0a72e005c1ad8,

title = "Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking",

abstract = "This paper investigates the trajectory tracking control method for a robotic manipulator with uncertainties. A compound controller combining a traditional control law with deep reinforcement learning is developed to improve tracking accuracy and adaptability. The model-based control method is able to increase sampling efficiency for learning control strategy. The introduction of deep reinforcement learning based on soft actor-critic structure and Lyapunov function constrain enables the system to compensate unknown uncertainties and remain stable. Eventually, a 3-DOF manipulator is used to show the effectiveness of the proposed controller. Comparative simulation results demonstrate that the compound controller acquires higher tracking accuracy than the pure model-based control method.",

keywords = "3-DOF, Deep reinforcement learning, trajectory tracking control, uncertain robotic manipulators",

author = "Aohua Liu and Bo Zhang and Weiliang Chen and Yiang Luo and Shuxian Fang and Ouyang Zhang and Zhuang Liu and Zhenhuan Wang and Jianxing Liu",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 Chinese Automation Congress, CAC 2022 ; Conference date: 25-11-2022 Through 27-11-2022",

year = "2022",

doi = "10.1109/CAC57257.2022.10055583",

language = "英语",

series = "Proceedings - 2022 Chinese Automation Congress, CAC 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2740--2745",

booktitle = "Proceedings - 2022 Chinese Automation Congress, CAC 2022",

address = "美国",

}

Liu, A, Zhang, B, Chen, W, Luo, Y, Fang, S, Zhang, O, Liu, Z , Wang, Z & Liu, J 2022, Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking. in Proceedings - 2022 Chinese Automation Congress, CAC 2022. Proceedings - 2022 Chinese Automation Congress, CAC 2022, vol. 2022-January, Institute of Electrical and Electronics Engineers Inc., pp. 2740-2745, 2022 Chinese Automation Congress, CAC 2022, Xiamen, China, 25/11/22. https://doi.org/10.1109/CAC57257.2022.10055583

Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking. / Liu, Aohua; Zhang, Bo; Chen, Weiliang et al.
Proceedings - 2022 Chinese Automation Congress, CAC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 2740-2745 (Proceedings - 2022 Chinese Automation Congress, CAC 2022; Vol. 2022-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking

AU - Liu, Aohua

AU - Zhang, Bo

AU - Chen, Weiliang

AU - Luo, Yiang

AU - Fang, Shuxian

AU - Zhang, Ouyang

AU - Liu, Zhuang

AU - Wang, Zhenhuan

AU - Liu, Jianxing

PY - 2022

Y1 - 2022

N2 - This paper investigates the trajectory tracking control method for a robotic manipulator with uncertainties. A compound controller combining a traditional control law with deep reinforcement learning is developed to improve tracking accuracy and adaptability. The model-based control method is able to increase sampling efficiency for learning control strategy. The introduction of deep reinforcement learning based on soft actor-critic structure and Lyapunov function constrain enables the system to compensate unknown uncertainties and remain stable. Eventually, a 3-DOF manipulator is used to show the effectiveness of the proposed controller. Comparative simulation results demonstrate that the compound controller acquires higher tracking accuracy than the pure model-based control method.

AB - This paper investigates the trajectory tracking control method for a robotic manipulator with uncertainties. A compound controller combining a traditional control law with deep reinforcement learning is developed to improve tracking accuracy and adaptability. The model-based control method is able to increase sampling efficiency for learning control strategy. The introduction of deep reinforcement learning based on soft actor-critic structure and Lyapunov function constrain enables the system to compensate unknown uncertainties and remain stable. Eventually, a 3-DOF manipulator is used to show the effectiveness of the proposed controller. Comparative simulation results demonstrate that the compound controller acquires higher tracking accuracy than the pure model-based control method.

KW - 3-DOF

KW - Deep reinforcement learning

KW - trajectory tracking control

KW - uncertain robotic manipulators

UR - https://www.scopus.com/pages/publications/85151287991

U2 - 10.1109/CAC57257.2022.10055583

DO - 10.1109/CAC57257.2022.10055583

M3 - 会议稿件

AN - SCOPUS:85151287991

T3 - Proceedings - 2022 Chinese Automation Congress, CAC 2022

SP - 2740

EP - 2745

BT - Proceedings - 2022 Chinese Automation Congress, CAC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 Chinese Automation Congress, CAC 2022

Y2 - 25 November 2022 through 27 November 2022

ER -

Liu A, Zhang B, Chen W, Luo Y, Fang S, Zhang O et al. Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking. In Proceedings - 2022 Chinese Automation Congress, CAC 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 2740-2745. (Proceedings - 2022 Chinese Automation Congress, CAC 2022). doi: 10.1109/CAC57257.2022.10055583

Reinforcement Learning Based Control for Uncertain Robotic Manipulator Trajectory Tracking

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this