Q_learning based on active backup and memory mechanism

Yang Liu; Mao Zu Guo; Hong Xun Yao

Q_learning based on active backup and memory mechanism

Yang Liu^*
, Mao Zu Guo
, Hong Xun Yao

^*Corresponding author for this work

School of Computer Science and Technology, Harbin Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Exploration is used in Q_learning because the agent will be caught in locally optimal policies due to blind exploitation. However excessive exploration will degrade the performance of Q_learning and it is difficult to meet the trade-off between exploration and exploitation. In this paper, the active backup is introduced into Q_learning and the corresponding algorithm AB_Q_learning based on Dijkstra backup in dynamic programming is proposed. Then, the memory mechanism based MEAB_Q_learning algorithm is given for the agent to learn in completely unknown environment. The experimental results show that these two algorithms not only converge more quickly, but also solve the problem of local optimization.

Original language	English
Title of host publication	Proceedings of 2004 International Conference on Machine Learning and Cybernetics
Pages	271-275
Number of pages	5
State	Published - 2004
Externally published	Yes
Event	Proceedings of 2004 International Conference on Machine Learning and Cybernetics - Shanghai, China Duration: 26 Aug 2004 → 29 Aug 2004

Publication series

Name	Proceedings of 2004 International Conference on Machine Learning and Cybernetics
Volume	1

Conference

Conference	Proceedings of 2004 International Conference on Machine Learning and Cybernetics
Country/Territory	China
City	Shanghai
Period	26/08/04 → 29/08/04

Keywords

Dijkstra backup
Q_learning
Reinforcement learning

Cite this

@inproceedings{0463996b87684c47a36f714c7ec44e76,

title = "Q\_learning based on active backup and memory mechanism",

abstract = "Exploration is used in Q\_learning because the agent will be caught in locally optimal policies due to blind exploitation. However excessive exploration will degrade the performance of Q\_learning and it is difficult to meet the trade-off between exploration and exploitation. In this paper, the active backup is introduced into Q\_learning and the corresponding algorithm AB\_Q\_learning based on Dijkstra backup in dynamic programming is proposed. Then, the memory mechanism based MEAB\_Q\_learning algorithm is given for the agent to learn in completely unknown environment. The experimental results show that these two algorithms not only converge more quickly, but also solve the problem of local optimization.",

keywords = "Dijkstra backup, Q\_learning, Reinforcement learning",

author = "Yang Liu and Guo, \{Mao Zu\} and Yao, \{Hong Xun\}",

year = "2004",

language = "英语",

isbn = "0780384032",

series = "Proceedings of 2004 International Conference on Machine Learning and Cybernetics",

pages = "271--275",

booktitle = "Proceedings of 2004 International Conference on Machine Learning and Cybernetics",

note = "Proceedings of 2004 International Conference on Machine Learning and Cybernetics ; Conference date: 26-08-2004 Through 29-08-2004",

}

Liu, Y, Guo, MZ & Yao, HX 2004, Q_learning based on active backup and memory mechanism. in Proceedings of 2004 International Conference on Machine Learning and Cybernetics. Proceedings of 2004 International Conference on Machine Learning and Cybernetics, vol. 1, pp. 271-275, Proceedings of 2004 International Conference on Machine Learning and Cybernetics, Shanghai, China, 26/08/04.

Q_learning based on active backup and memory mechanism. / Liu, Yang; Guo, Mao Zu; Yao, Hong Xun.
Proceedings of 2004 International Conference on Machine Learning and Cybernetics. 2004. p. 271-275 (Proceedings of 2004 International Conference on Machine Learning and Cybernetics; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Q_learning based on active backup and memory mechanism

AU - Liu, Yang

AU - Guo, Mao Zu

AU - Yao, Hong Xun

PY - 2004

Y1 - 2004

N2 - Exploration is used in Q_learning because the agent will be caught in locally optimal policies due to blind exploitation. However excessive exploration will degrade the performance of Q_learning and it is difficult to meet the trade-off between exploration and exploitation. In this paper, the active backup is introduced into Q_learning and the corresponding algorithm AB_Q_learning based on Dijkstra backup in dynamic programming is proposed. Then, the memory mechanism based MEAB_Q_learning algorithm is given for the agent to learn in completely unknown environment. The experimental results show that these two algorithms not only converge more quickly, but also solve the problem of local optimization.

AB - Exploration is used in Q_learning because the agent will be caught in locally optimal policies due to blind exploitation. However excessive exploration will degrade the performance of Q_learning and it is difficult to meet the trade-off between exploration and exploitation. In this paper, the active backup is introduced into Q_learning and the corresponding algorithm AB_Q_learning based on Dijkstra backup in dynamic programming is proposed. Then, the memory mechanism based MEAB_Q_learning algorithm is given for the agent to learn in completely unknown environment. The experimental results show that these two algorithms not only converge more quickly, but also solve the problem of local optimization.

KW - Dijkstra backup

KW - Q_learning

KW - Reinforcement learning

UR - https://www.scopus.com/pages/publications/6344277095

M3 - 会议稿件

AN - SCOPUS:6344277095

SN - 0780384032

T3 - Proceedings of 2004 International Conference on Machine Learning and Cybernetics

SP - 271

EP - 275

BT - Proceedings of 2004 International Conference on Machine Learning and Cybernetics

T2 - Proceedings of 2004 International Conference on Machine Learning and Cybernetics

Y2 - 26 August 2004 through 29 August 2004

ER -

Q_learning based on active backup and memory mechanism

Abstract

Publication series

Conference

Keywords

Other files and links

Fingerprint

Cite this