Q-learning based network selection for WCDMA/WLAN heterogeneous wireless networks

This paper investigates the problem of network selection in access control for heterogeneous networks of WCDMA/WLAN. To optimize the network selection decision policy, an optimization equation is defined with the objective of maximizing the total rewards, and then a new Q-learning Based Network Selection (QBNS) mechanism is proposed to solve the equation. In the QBNS Algorithm, Q-learning algorithm is employed by taking both of the network capacity and the quality of service (QoS) requirements of users into account. In addition, the network states are analyzed by considering interference power of WCDMA subnet and channel busyness ratio of WLAN subnet. Simulation results show that the proposed QBNS scheme can obtain lower call blocking probability and much higher total reward performance than the traditional Semi-Markov Decision Process (SMDP) Algorithm.