A strategic learning algorithm for state-based games

Learning algorithm design and applications of state-based games are investigated. First, a heuristic uncoupled learning algorithm, which is a two memory better reply learning rule, is proposed. Under reachability conditions it is proved that for any initial state, if all agents in the state-based game follow the proposed learning algorithm, the action state pair converges almost surely to an action invariant set of recurrent state equilibria. The design of the learning algorithm relies on global and local searches with finite memory, inertia, and randomness. Then, existence of time-efficient universal learning algorithm is studied. Finally, applications of our proposed learning algorithm are discussed, including learning pure Nash equilibrium in finite games and cooperative control with time-varying communication structure.