Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning

Shuo Shi; Meng Wang; Xuemai Gu

doi:10.1007/978-3-030-69066-3_26

Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning

Shuo Shi^*
, Meng Wang
, Xuemai Gu

^*Corresponding author for this work

School of Electronics and Information Engineering, Harbin Institute of Technology
Peng Cheng Laboratory
International Innovation Institute of HIT in Huizhou

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Industrial Internet of things (IIoT) has been envisioned as a key technology for Industry 4.0. However, the battery capcity and processing ability of IIoT devices are limited which imposes great challenges when handling tasks with high quality of service (QoS) requirements. Toward this end, in this paper we first use multiple unmanned aerial vehicles (UAVs) equipped with computation resources to offer computation offloading opportunities for IIoT devices due to their high flexibility. Then we formulate the multi-UAV-enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. Furthermore, to obtain the energy-efficient solutions for IIoT devices, we propose an intelligent algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Simulation results show that the proposed MDSPR converges fast and outperforms the normal deep Q-learning (DQN) method and other benchmark algorithms in terms of energy-efficiency and tasks’ successful rate.

Original language	English
Title of host publication	Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings
Editors	Shuo Shi, Liang Ye, Yu Zhang
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	295-305
Number of pages	11
ISBN (Print)	9783030690656
DOIs	https://doi.org/10.1007/978-3-030-69066-3_26
State	Published - 2021
Externally published	Yes
Event	2nd EAI International Conference on Artificial Intelligence for Communications and Networks, AICON 2020 - Harbin, China Duration: 19 Dec 2020 → 20 Dec 2020

Publication series

Name	Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume	356 LNICST
ISSN (Print)	1867-8211
ISSN (Electronic)	1867-822X

Conference

Conference	2nd EAI International Conference on Artificial Intelligence for Communications and Networks, AICON 2020
Country/Territory	China
City	Harbin
Period	19/12/20 → 20/12/20

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

Computation offloading
Deep reinforcement learning
UAV-enabled IIoT

Access to Document

10.1007/978-3-030-69066-3_26

Cite this

Shi, S., Wang, M., & Gu, X. (2021). Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning. In S. Shi, L. Ye, & Y. Zhang (Eds.), Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings (pp. 295-305). (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 356 LNICST). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-69066-3_26

Shi, Shuo ; Wang, Meng ; Gu, Xuemai. / Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning. Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings. editor / Shuo Shi ; Liang Ye ; Yu Zhang. Springer Science and Business Media Deutschland GmbH, 2021. pp. 295-305 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST).

@inproceedings{6ad22271e4574676b46b5fa724c101ee,

title = "Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning",

abstract = "Industrial Internet of things (IIoT) has been envisioned as a key technology for Industry 4.0. However, the battery capcity and processing ability of IIoT devices are limited which imposes great challenges when handling tasks with high quality of service (QoS) requirements. Toward this end, in this paper we first use multiple unmanned aerial vehicles (UAVs) equipped with computation resources to offer computation offloading opportunities for IIoT devices due to their high flexibility. Then we formulate the multi-UAV-enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. Furthermore, to obtain the energy-efficient solutions for IIoT devices, we propose an intelligent algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Simulation results show that the proposed MDSPR converges fast and outperforms the normal deep Q-learning (DQN) method and other benchmark algorithms in terms of energy-efficiency and tasks{\textquoteright} successful rate.",

keywords = "Computation offloading, Deep reinforcement learning, UAV-enabled IIoT",

author = "Shuo Shi and Meng Wang and Xuemai Gu",

note = "Publisher Copyright: {\textcopyright} 2021, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.; 2nd EAI International Conference on Artificial Intelligence for Communications and Networks, AICON 2020 ; Conference date: 19-12-2020 Through 20-12-2020",

year = "2021",

doi = "10.1007/978-3-030-69066-3\_26",

language = "英语",

isbn = "9783030690656",

series = "Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "295--305",

editor = "Shuo Shi and Liang Ye and Yu Zhang",

booktitle = "Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings",

address = "德国",

}

Shi, S, Wang, M & Gu, X 2021, Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning. in S Shi, L Ye & Y Zhang (eds), Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 356 LNICST, Springer Science and Business Media Deutschland GmbH, pp. 295-305, 2nd EAI International Conference on Artificial Intelligence for Communications and Networks, AICON 2020, Harbin, China, 19/12/20. https://doi.org/10.1007/978-3-030-69066-3_26

Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning. / Shi, Shuo; Wang, Meng; Gu, Xuemai.
Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings. ed. / Shuo Shi; Liang Ye; Yu Zhang. Springer Science and Business Media Deutschland GmbH, 2021. p. 295-305 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 356 LNICST).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning

AU - Shi, Shuo

AU - Wang, Meng

AU - Gu, Xuemai

PY - 2021

Y1 - 2021

N2 - Industrial Internet of things (IIoT) has been envisioned as a key technology for Industry 4.0. However, the battery capcity and processing ability of IIoT devices are limited which imposes great challenges when handling tasks with high quality of service (QoS) requirements. Toward this end, in this paper we first use multiple unmanned aerial vehicles (UAVs) equipped with computation resources to offer computation offloading opportunities for IIoT devices due to their high flexibility. Then we formulate the multi-UAV-enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. Furthermore, to obtain the energy-efficient solutions for IIoT devices, we propose an intelligent algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Simulation results show that the proposed MDSPR converges fast and outperforms the normal deep Q-learning (DQN) method and other benchmark algorithms in terms of energy-efficiency and tasks’ successful rate.

AB - Industrial Internet of things (IIoT) has been envisioned as a key technology for Industry 4.0. However, the battery capcity and processing ability of IIoT devices are limited which imposes great challenges when handling tasks with high quality of service (QoS) requirements. Toward this end, in this paper we first use multiple unmanned aerial vehicles (UAVs) equipped with computation resources to offer computation offloading opportunities for IIoT devices due to their high flexibility. Then we formulate the multi-UAV-enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. Furthermore, to obtain the energy-efficient solutions for IIoT devices, we propose an intelligent algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Simulation results show that the proposed MDSPR converges fast and outperforms the normal deep Q-learning (DQN) method and other benchmark algorithms in terms of energy-efficiency and tasks’ successful rate.

KW - Computation offloading

KW - Deep reinforcement learning

KW - UAV-enabled IIoT

UR - https://www.scopus.com/pages/publications/85104468505

U2 - 10.1007/978-3-030-69066-3_26

DO - 10.1007/978-3-030-69066-3_26

M3 - 会议稿件

AN - SCOPUS:85104468505

SN - 9783030690656

T3 - Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

SP - 295

EP - 305

BT - Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings

A2 - Shi, Shuo

A2 - Ye, Liang

A2 - Zhang, Yu

PB - Springer Science and Business Media Deutschland GmbH

T2 - 2nd EAI International Conference on Artificial Intelligence for Communications and Networks, AICON 2020

Y2 - 19 December 2020 through 20 December 2020

ER -

Shi S, Wang M, Gu X. Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning. In Shi S, Ye L, Zhang Y, editors, Artificial Intelligence for Communications and Networks - 2nd EAI International Conference, AICON 2020, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 295-305. (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST). doi: 10.1007/978-3-030-69066-3_26

Energy-Efficient Multi-UAV-Enabled Computation Offloading for Industrial Internet of Things via Deep Reinforcement Learning

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this