Reinforcement learning function approximation algorithm based on linear average

A reinforcement learning algorithm based on linear average is proposed, which is used to solve non-convergent problems of reinforcement learning function approximation in continuous state space. According to contraction theory, this algorithm is based on gradient descent method, which adopts linear average as performance evaluation of value function. So the iterative process of value function becomes a convergent process to a fixed value. A standard reinforcement learning problem, Mountain Car Problem, is used to verify the performance of the algorithm. Results show the effectiveness, feasibility and quick convergence of the algorithm.