Iterative Doubly-spread Channel Estimation based on Reinforcement Learning for Underwater Acoustic Communication

Underwater acoustic (UWA) channels are usually featured with long delay spreads, significant Doppler effects and time-varying nature, due to internal waves, platform and sea-surface motion. Reinforcement learning (RL) is a feedback-based machine learning technique where an intelligent agent (computer program) can perceive and interpret the environment, take actions and learn through trials and errors. It motives us to use the RL to perceive the underwater acoustic environment, thus estimate the variation of the UWA channels. Therefore, we propose a channel estimation method based on the RL, we can estimate the time-varying Doppler scaler and multipath sparsity through interacting with the UWA channels with an iterative structure. Experimental results demonstrate the performance superiority of the proposed method over existing channel estimation methods.