Timely Remote Control in Wireless Cyber-Physical System With Multiple Processes: A Cross-Time Slot Scheduling Policy

This paper investigates a wireless remote control problem in cyber-physical system (CPS) with multiple processes. In the system, sensors collect the state information of each process and transmit it to the controller through wireless channels. The communication constraints, including transmission delays, packet loss, and bandwidth limitation, are taken into account. To evaluate control timeliness for each process, this paper adopts the concept of age of information (AoI) in the context of closed-loop control. Meanwhile, to strike a tradeoff between transmission delay and outage, this paper introduces an innovative cross-slot scheduling policy not covered in existing literature, which can freely allocate transmission time and occupancy bandwidth. We prove that the scheduling problem in bandwidth limited remote control is an NP-hard problem, and establish the optimization problem as a Markov decision process (MDP) problem. The Deep-Double-Dueling-Q-Learning (D3QN) algorithm is employed to approximate the optimal scheduling policy for the scenario where the channel information is unknown and the system is model-free. By extensive simulations, the proposed cross-time slot scheduling policy demonstrates superior effectiveness in allocating time-frequency resources and achieving outstanding results.