Efficient Beacon-Aided AUV Localization: A Reinforcement Learning Based Approach

Beacon-aided autonomous underwater vehicle (AUV) localization supporting maritime surveillance applications in underwater acoustic sensor networks selects a fixed number of beacons with constant transmit power, and thus has degradation of localization accuracy with severe channel fading and position fluctuation of beacons. In this paper, we propose a reinforcement learning based AUV localization scheme to choose the beacons and their transmit power to improve the localization accuracy and energy efficiency based on the AUV depth, the received signal strength, the number of selected beacons and the beacon energy consumption. According to the least squares method, the AUV position is calculated based on the isogradient sound speed model and the round-trip time of the localization signals. The localization error averaged over different beacon sets is evaluated to formulate the localization policy distribution. Deep neural network is designed to estimate the expected long-term discounted utility with higher feature extraction efficiency for the underwater networks with a large number of beacons. The Cramer-Rao lower bounds of the proposed localization schemes are derived to analyze the effect of the position fluctuation of beacons on the localization accuracy. Simulation results verify the performance gain in terms of the localization accuracy and the beacon energy consumption over the benchmark.