An approximate dynamic programming approach for model-free control of switched systems

Several approximate dynamic programming (ADP) algorithms have been developed and demonstrated for the model-free control of continuous and discrete dynamical systems. However, their applicability to hybrid systems that involve both discrete and continuous state and control variables has yet to be demonstrated in the literature. This paper presents an ADP approach for hy- brid systems (hybrid-ADP) that obtains the optimal control law and discrete action sequence via online learning. New recursive relationships for hybrid-ADP are presented for switched hybrid systems that are possibly nonlinear. In order to demonstrate the ability of the proposed ADP algorithm to converge to the optimal solution, the approach is demonstrated on a switched, linear hybrid system with a quadratic cost function, for which there exists an analytical solution. The results show that the ADP algorithm is capable of converging to the optimal switched control law, by minimizing the cost-to-go online, based on an observable state vector.