Sample-efficient policy learning based on completely behavior cloning

Qiming Zou; Ling Wang; Yu Li; Jie Liu

doi:10.1109/SMC.2019.8914085

Sample-efficient policy learning based on completely behavior cloning

Qiming Zou
, Ling Wang
, Yu Li
, Jie Liu

Faculty of Computing

Harbin Institute of Technology
Anhui University of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Direct policy search is one of the most important algorithm of reinforcement learning. However, learning from scratch needs a large amount of experience data and can be easily prone to poor local optima. In order to overcome these challenges, this paper proposed a training-free behavior cloning algorithm called Policy Learning based on Completely Behavior Cloning (PLCBC). PLCBC transforms the Model Predictive Control (MPC) controller into a PieceWise Affine (PWA) function with multi-parametric programming, and uses a neural network to express this function. By this way, off-the-shelf deep reinforcement learning algorithms can be used to fine-tune this neural network. The experiments show that our method can help agent learn at the high reward state region, and converge faster and better.

Original language	English
Title of host publication	2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2543-2548
Number of pages	6
ISBN (Electronic)	9781728145693
DOIs	https://doi.org/10.1109/SMC.2019.8914085
State	Published - Oct 2019
Event	2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019 - Bari, Italy Duration: 6 Oct 2019 → 9 Oct 2019

Publication series

Name	Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
Volume	2019-October
ISSN (Print)	1062-922X

Conference

Conference	2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019
Country/Territory	Italy
City	Bari
Period	6/10/19 → 9/10/19

Access to Document

10.1109/SMC.2019.8914085

Cite this

Zou, Q., Wang, L., Li, Y., & Liu, J. (2019). Sample-efficient policy learning based on completely behavior cloning. In 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019 (pp. 2543-2548). Article 8914085 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics; Vol. 2019-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2019.8914085

@inproceedings{5bf1b8e3d3344462bef89b9a602d789c,

title = "Sample-efficient policy learning based on completely behavior cloning",

abstract = "Direct policy search is one of the most important algorithm of reinforcement learning. However, learning from scratch needs a large amount of experience data and can be easily prone to poor local optima. In order to overcome these challenges, this paper proposed a training-free behavior cloning algorithm called Policy Learning based on Completely Behavior Cloning (PLCBC). PLCBC transforms the Model Predictive Control (MPC) controller into a PieceWise Affine (PWA) function with multi-parametric programming, and uses a neural network to express this function. By this way, off-the-shelf deep reinforcement learning algorithms can be used to fine-tune this neural network. The experiments show that our method can help agent learn at the high reward state region, and converge faster and better.",

author = "Qiming Zou and Ling Wang and Yu Li and Jie Liu",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019 ; Conference date: 06-10-2019 Through 09-10-2019",

year = "2019",

month = oct,

doi = "10.1109/SMC.2019.8914085",

language = "英语",

series = "Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2543--2548",

booktitle = "2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019",

address = "美国",

}

Zou, Q, Wang, L, Li, Y & Liu, J 2019, Sample-efficient policy learning based on completely behavior cloning. in 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019., 8914085, Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, vol. 2019-October, Institute of Electrical and Electronics Engineers Inc., pp. 2543-2548, 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019, Bari, Italy, 6/10/19. https://doi.org/10.1109/SMC.2019.8914085

Sample-efficient policy learning based on completely behavior cloning. / Zou, Qiming; Wang, Ling; Li, Yu et al.
2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 2543-2548 8914085 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics; Vol. 2019-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Sample-efficient policy learning based on completely behavior cloning

AU - Zou, Qiming

AU - Wang, Ling

AU - Li, Yu

AU - Liu, Jie

PY - 2019/10

Y1 - 2019/10

N2 - Direct policy search is one of the most important algorithm of reinforcement learning. However, learning from scratch needs a large amount of experience data and can be easily prone to poor local optima. In order to overcome these challenges, this paper proposed a training-free behavior cloning algorithm called Policy Learning based on Completely Behavior Cloning (PLCBC). PLCBC transforms the Model Predictive Control (MPC) controller into a PieceWise Affine (PWA) function with multi-parametric programming, and uses a neural network to express this function. By this way, off-the-shelf deep reinforcement learning algorithms can be used to fine-tune this neural network. The experiments show that our method can help agent learn at the high reward state region, and converge faster and better.

AB - Direct policy search is one of the most important algorithm of reinforcement learning. However, learning from scratch needs a large amount of experience data and can be easily prone to poor local optima. In order to overcome these challenges, this paper proposed a training-free behavior cloning algorithm called Policy Learning based on Completely Behavior Cloning (PLCBC). PLCBC transforms the Model Predictive Control (MPC) controller into a PieceWise Affine (PWA) function with multi-parametric programming, and uses a neural network to express this function. By this way, off-the-shelf deep reinforcement learning algorithms can be used to fine-tune this neural network. The experiments show that our method can help agent learn at the high reward state region, and converge faster and better.

UR - https://www.scopus.com/pages/publications/85076737767

U2 - 10.1109/SMC.2019.8914085

DO - 10.1109/SMC.2019.8914085

M3 - 会议稿件

AN - SCOPUS:85076737767

T3 - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

SP - 2543

EP - 2548

BT - 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019

Y2 - 6 October 2019 through 9 October 2019

ER -

Zou Q, Wang L, Li Y, Liu J. Sample-efficient policy learning based on completely behavior cloning. In 2019 IEEE International Conference on Systems, Man and Cybernetics, SMC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 2543-2548. 8914085. (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). doi: 10.1109/SMC.2019.8914085

Sample-efficient policy learning based on completely behavior cloning

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this