RVI reinforcement learning for Semi-Markov decision processes with average reward

Yanjie Li; Fang Cao

doi:10.1109/WCICA.2010.5554785

RVI reinforcement learning for Semi-Markov decision processes with average reward

Yanjie Li^*
, Fang Cao

^*Corresponding author for this work

Harbin Institute of Technology Shenzhen
Beijing Jiaotong University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Based on the sensitivity-based approach, we discuss the reinforcement learning problem of semi-Markov decision processes (SMDPs) with average reward. First, we provide a new Bellman optimality equation. On this basis, we propose a relative value iteration (RVI) reinforcement learning algorithm. The new RVI reinforcement learning algorithm may avoid the estimation of optimal average reward in the process of learning and has a good convergence rate.

Original language	English
Title of host publication	2010 8th World Congress on Intelligent Control and Automation, WCICA 2010
Pages	1674-1679
Number of pages	6
DOIs	https://doi.org/10.1109/WCICA.2010.5554785
State	Published - 2010
Externally published	Yes
Event	2010 8th World Congress on Intelligent Control and Automation, WCICA 2010 - Jinan, China Duration: 7 Jul 2010 → 9 Jul 2010

Publication series

Name	Proceedings of the World Congress on Intelligent Control and Automation (WCICA)

Conference

Conference	2010 8th World Congress on Intelligent Control and Automation, WCICA 2010
Country/Territory	China
City	Jinan
Period	7/07/10 → 9/07/10

Keywords

Performance potential
Reinforcement learning
Relative value iteration
Semi-Markov decision processes

Access to Document

10.1109/WCICA.2010.5554785

Cite this

@inproceedings{a8288c88d8134f1f8d3d27171fb492c1,

title = "RVI reinforcement learning for Semi-Markov decision processes with average reward",

abstract = "Based on the sensitivity-based approach, we discuss the reinforcement learning problem of semi-Markov decision processes (SMDPs) with average reward. First, we provide a new Bellman optimality equation. On this basis, we propose a relative value iteration (RVI) reinforcement learning algorithm. The new RVI reinforcement learning algorithm may avoid the estimation of optimal average reward in the process of learning and has a good convergence rate.",

keywords = "Performance potential, Reinforcement learning, Relative value iteration, Semi-Markov decision processes",

author = "Yanjie Li and Fang Cao",

year = "2010",

doi = "10.1109/WCICA.2010.5554785",

language = "英语",

isbn = "9781424467129",

series = "Proceedings of the World Congress on Intelligent Control and Automation (WCICA)",

pages = "1674--1679",

booktitle = "2010 8th World Congress on Intelligent Control and Automation, WCICA 2010",

note = "2010 8th World Congress on Intelligent Control and Automation, WCICA 2010 ; Conference date: 07-07-2010 Through 09-07-2010",

}

Li, Y & Cao, F 2010, RVI reinforcement learning for Semi-Markov decision processes with average reward. in 2010 8th World Congress on Intelligent Control and Automation, WCICA 2010., 5554785, Proceedings of the World Congress on Intelligent Control and Automation (WCICA), pp. 1674-1679, 2010 8th World Congress on Intelligent Control and Automation, WCICA 2010, Jinan, China, 7/07/10. https://doi.org/10.1109/WCICA.2010.5554785

RVI reinforcement learning for Semi-Markov decision processes with average reward. / Li, Yanjie; Cao, Fang.
2010 8th World Congress on Intelligent Control and Automation, WCICA 2010. 2010. p. 1674-1679 5554785 (Proceedings of the World Congress on Intelligent Control and Automation (WCICA)).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - RVI reinforcement learning for Semi-Markov decision processes with average reward

AU - Li, Yanjie

AU - Cao, Fang

PY - 2010

Y1 - 2010

N2 - Based on the sensitivity-based approach, we discuss the reinforcement learning problem of semi-Markov decision processes (SMDPs) with average reward. First, we provide a new Bellman optimality equation. On this basis, we propose a relative value iteration (RVI) reinforcement learning algorithm. The new RVI reinforcement learning algorithm may avoid the estimation of optimal average reward in the process of learning and has a good convergence rate.

AB - Based on the sensitivity-based approach, we discuss the reinforcement learning problem of semi-Markov decision processes (SMDPs) with average reward. First, we provide a new Bellman optimality equation. On this basis, we propose a relative value iteration (RVI) reinforcement learning algorithm. The new RVI reinforcement learning algorithm may avoid the estimation of optimal average reward in the process of learning and has a good convergence rate.

KW - Performance potential

KW - Reinforcement learning

KW - Relative value iteration

KW - Semi-Markov decision processes

UR - https://www.scopus.com/pages/publications/77958114662

U2 - 10.1109/WCICA.2010.5554785

DO - 10.1109/WCICA.2010.5554785

M3 - 会议稿件

AN - SCOPUS:77958114662

SN - 9781424467129

T3 - Proceedings of the World Congress on Intelligent Control and Automation (WCICA)

SP - 1674

EP - 1679

BT - 2010 8th World Congress on Intelligent Control and Automation, WCICA 2010

T2 - 2010 8th World Congress on Intelligent Control and Automation, WCICA 2010

Y2 - 7 July 2010 through 9 July 2010

ER -

RVI reinforcement learning for Semi-Markov decision processes with average reward

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this