A unified approach to time-aggregated Markov decision processes

Yanjie Li; Xinyu Wu

doi:10.1016/j.automatica.2015.12.022

A unified approach to time-aggregated Markov decision processes

Yanjie Li
, Xinyu Wu

Harbin Institute of Technology Shenzhen
Shenzhen Institute of Advanced Technology

Research output: Contribution to journal › Article › peer-review

7 Scopus citations

Abstract

This paper presents a unified approach to time-aggregated Markov decision processes (MDPs) with an average cost criterion. The approach is based on a framework in which a time-aggregated MDP constitutes a semi-Markov decision process (SMDP). By analyzing the performance sensitivity formulas of this SMDP, a number of optimization algorithms for time aggregated MDPs, including those previously reported in the literature, can be developed in a simple and intuitive way.

Original language	English
Pages (from-to)	77-84
Number of pages	8
Journal	Automatica
Volume	67
DOIs	https://doi.org/10.1016/j.automatica.2015.12.022
State	Published - 1 May 2016
Externally published	Yes

Keywords

Markov decision process
Performance sensitivity
Semi-Markov decision process
Time aggregation

Access to Document

10.1016/j.automatica.2015.12.022

Cite this

@article{467597c842b94675ab48811e64b68787,

title = "A unified approach to time-aggregated Markov decision processes",

abstract = "This paper presents a unified approach to time-aggregated Markov decision processes (MDPs) with an average cost criterion. The approach is based on a framework in which a time-aggregated MDP constitutes a semi-Markov decision process (SMDP). By analyzing the performance sensitivity formulas of this SMDP, a number of optimization algorithms for time aggregated MDPs, including those previously reported in the literature, can be developed in a simple and intuitive way.",

keywords = "Markov decision process, Performance sensitivity, Semi-Markov decision process, Time aggregation",

author = "Yanjie Li and Xinyu Wu",

year = "2016",

month = may,

day = "1",

doi = "10.1016/j.automatica.2015.12.022",

language = "英语",

volume = "67",

pages = "77--84",

journal = "Automatica",

issn = "0005-1098",

publisher = "Elsevier Ltd",

}

TY - JOUR

T1 - A unified approach to time-aggregated Markov decision processes

AU - Li, Yanjie

AU - Wu, Xinyu

PY - 2016/5/1

Y1 - 2016/5/1

N2 - This paper presents a unified approach to time-aggregated Markov decision processes (MDPs) with an average cost criterion. The approach is based on a framework in which a time-aggregated MDP constitutes a semi-Markov decision process (SMDP). By analyzing the performance sensitivity formulas of this SMDP, a number of optimization algorithms for time aggregated MDPs, including those previously reported in the literature, can be developed in a simple and intuitive way.

AB - This paper presents a unified approach to time-aggregated Markov decision processes (MDPs) with an average cost criterion. The approach is based on a framework in which a time-aggregated MDP constitutes a semi-Markov decision process (SMDP). By analyzing the performance sensitivity formulas of this SMDP, a number of optimization algorithms for time aggregated MDPs, including those previously reported in the literature, can be developed in a simple and intuitive way.

KW - Markov decision process

KW - Performance sensitivity

KW - Semi-Markov decision process

KW - Time aggregation

UR - https://www.scopus.com/pages/publications/84960345301

U2 - 10.1016/j.automatica.2015.12.022

DO - 10.1016/j.automatica.2015.12.022

M3 - 文章

AN - SCOPUS:84960345301

SN - 0005-1098

VL - 67

SP - 77

EP - 84

JO - Automatica

JF - Automatica

ER -

A unified approach to time-aggregated Markov decision processes

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this