Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning

Desong Du; Qihang Zhou; Naiming Qi; Xu Wang; Yanfang Liu

doi:10.1109/ICUS48101.2019.8995991

Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning

Desong Du
, Qihang Zhou
, Naiming Qi
, Xu Wang
, Yanfang Liu

School of Astronautics, Harbin Institute of Technology
China Aerospace Science and Technology Corporation

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

23 Scopus citations

Abstract

With the complexity of the dynamic model of free-floating space robots (FFSR), it is difficult to design the control system to capture targets. This paper presents a controller for FFSR to capture targets without the kinematic and dynamic model equations, where the agent learns a closed-loop control policy from state information only. At first, the process of the task is described as the reinforcement learning process without the dynamic models of the space robot. Then, we use the deep deterministic policy algorithm (DDPG) to train the policy for space manipulator motion planning. And we present a skill named "pre-training" in the training process to further import the learning efficiency. Finally, a 3 degrees of freedom space robot is modeled and simulated to demonstrate the validity of the controller.

Original language	English
Title of host publication	Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	519-523
Number of pages	5
ISBN (Electronic)	9781728137926
DOIs	https://doi.org/10.1109/ICUS48101.2019.8995991
State	Published - Oct 2019
Externally published	Yes
Event	2019 IEEE International Conference on Unmanned Systems, ICUS 2019 - Beijing, China Duration: 17 Oct 2019 → 19 Oct 2019

Publication series

Name	Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019

Conference

Conference	2019 IEEE International Conference on Unmanned Systems, ICUS 2019
Country/Territory	China
City	Beijing
Period	17/10/19 → 19/10/19

Keywords

deep reinforcement learning
free-floating space robot
motion planning

Access to Document

10.1109/ICUS48101.2019.8995991

Cite this

Du, D., Zhou, Q., Qi, N., Wang, X., & Liu, Y. (2019). Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning. In Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019 (pp. 519-523). Article 8995991 (Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICUS48101.2019.8995991

Du, Desong ; Zhou, Qihang ; Qi, Naiming et al. / Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning. Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 519-523 (Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019).

@inproceedings{bdda12615e524d04b6424207092b8c26,

title = "Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning",

abstract = "With the complexity of the dynamic model of free-floating space robots (FFSR), it is difficult to design the control system to capture targets. This paper presents a controller for FFSR to capture targets without the kinematic and dynamic model equations, where the agent learns a closed-loop control policy from state information only. At first, the process of the task is described as the reinforcement learning process without the dynamic models of the space robot. Then, we use the deep deterministic policy algorithm (DDPG) to train the policy for space manipulator motion planning. And we present a skill named {"}pre-training{"} in the training process to further import the learning efficiency. Finally, a 3 degrees of freedom space robot is modeled and simulated to demonstrate the validity of the controller.",

keywords = "deep reinforcement learning, free-floating space robot, motion planning",

author = "Desong Du and Qihang Zhou and Naiming Qi and Xu Wang and Yanfang Liu",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE International Conference on Unmanned Systems, ICUS 2019 ; Conference date: 17-10-2019 Through 19-10-2019",

year = "2019",

month = oct,

doi = "10.1109/ICUS48101.2019.8995991",

language = "英语",

series = "Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "519--523",

booktitle = "Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019",

address = "美国",

}

Du, D, Zhou, Q, Qi, N, Wang, X & Liu, Y 2019, Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning. in Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019., 8995991, Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019, Institute of Electrical and Electronics Engineers Inc., pp. 519-523, 2019 IEEE International Conference on Unmanned Systems, ICUS 2019, Beijing, China, 17/10/19. https://doi.org/10.1109/ICUS48101.2019.8995991

Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning. / Du, Desong; Zhou, Qihang; Qi, Naiming et al.
Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 519-523 8995991 (Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning

AU - Du, Desong

AU - Zhou, Qihang

AU - Qi, Naiming

AU - Wang, Xu

AU - Liu, Yanfang

PY - 2019/10

Y1 - 2019/10

N2 - With the complexity of the dynamic model of free-floating space robots (FFSR), it is difficult to design the control system to capture targets. This paper presents a controller for FFSR to capture targets without the kinematic and dynamic model equations, where the agent learns a closed-loop control policy from state information only. At first, the process of the task is described as the reinforcement learning process without the dynamic models of the space robot. Then, we use the deep deterministic policy algorithm (DDPG) to train the policy for space manipulator motion planning. And we present a skill named "pre-training" in the training process to further import the learning efficiency. Finally, a 3 degrees of freedom space robot is modeled and simulated to demonstrate the validity of the controller.

AB - With the complexity of the dynamic model of free-floating space robots (FFSR), it is difficult to design the control system to capture targets. This paper presents a controller for FFSR to capture targets without the kinematic and dynamic model equations, where the agent learns a closed-loop control policy from state information only. At first, the process of the task is described as the reinforcement learning process without the dynamic models of the space robot. Then, we use the deep deterministic policy algorithm (DDPG) to train the policy for space manipulator motion planning. And we present a skill named "pre-training" in the training process to further import the learning efficiency. Finally, a 3 degrees of freedom space robot is modeled and simulated to demonstrate the validity of the controller.

KW - deep reinforcement learning

KW - free-floating space robot

KW - motion planning

UR - https://www.scopus.com/pages/publications/85080957105

U2 - 10.1109/ICUS48101.2019.8995991

DO - 10.1109/ICUS48101.2019.8995991

M3 - 会议稿件

AN - SCOPUS:85080957105

T3 - Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019

SP - 519

EP - 523

BT - Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE International Conference on Unmanned Systems, ICUS 2019

Y2 - 17 October 2019 through 19 October 2019

ER -

Du D, Zhou Q, Qi N, Wang X, Liu Y. Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning. In Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 519-523. 8995991. (Proceedings of the 2019 IEEE International Conference on Unmanned Systems, ICUS 2019). doi: 10.1109/ICUS48101.2019.8995991

Learning to Control a Free-floating Space Robot using Deep Reinforcement Learning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this