An adaptive accelerated derivative-free optimization algorithm based on noncommutative maps

In this article, an adaptive accelerated derivative-free optimization algorithm is developed. A composition of noncommutative maps based on objective function evaluations is used to approximate an accelerated gradient descent algorithm with a momentum term. An adaptive step-size rule and an adaptive momentum term are introduced to improve the algorithm’s performance in terms of convergence speed and steady-state accuracy. Semi-global asymptotic stability of the proposed algorithm is proved for a class of convex objective functions under suitable assumptions. Simulation results are shown and compared to other derivative-free optimization algorithms.