Clustering state membership-based Q-learning for dynamic scheduling

Q-learning was applied to resolution of the adaptive dispatching rule selection problem under dynamic single-machine scheduling environment. Considering that Q-learning is hard to converge due to the large scale of the system state space during dynamic scheduling, the method extracts several state features of the system firstly, so that the dimension of the system state space can be reduced through the fuzzy clustering method. Then the machine agent can choose proper rules based on the transient system state membership of all the clustering system states. Each time after machine agent performs an action, the reward is assigned to all the value functions of the same rule in different clustering system states according to the fuzzy membership. The simulation results demonstrate that the proposed algorithm has a faster convergence rate, compared with the traditional Q-learning algorithm, and can improve the dynamic dispatching rule selection ability of machine agent.