Path planning based on improved deep Q-network algorithm for bionic robotic fish with ocean currents

Path planning is a fundamental issue for bionic robotic fish (BRF) to complete the missions such as underwater resource exploration, cable inspection, water quality detection and so on. The existing challenge is that how to design an efficient optimal path with uncertainty of ocean currents for the BRF working in the complex underwater three-dimensional (3D) environment. This paper proposes an improved deep Q-network (DQN) algorithm to improve the ability of BRF path planning in the unknown complex environment with dynamic ocean currents. A dynamic integrated reward mechanism is proposed based on the environmental information of terrains, obstacles, ocean currents and so on, which is used to balance multiple objective functions for the path planning problem. It presents a dynamic two-step action-selection strategy for the improved DQN algorithm. For avoiding premature convergence to the suboptimal path in the early stage, Boltzmann strategy is used to balance high-value actions and new actions in the first step; considering the factors such as ocean currents in the later stage, ϵ-greedy strategy is employed to exploit known information efficiently and design the path in the second step. A double dynamic learning rate method is proposed in light of the meta-gradient descent and Adam optimization methods, which is used to optimize the learning rate at two levels of hyperparameter and model parameter. Simulation results demonstrate the effectiveness of the proposed algorithm for the BRF path planning in the complex environment with dynamic ocean currents.