Maneuvering penetration strategies of ballistic missiles based on deep reinforcement learning

In this paper, a ballistic missile terminal penetration scenario is studied, which contains three participants: target, missile, and defender. The ballistic missile attempts to hit the target while evading the defender. A maneuvering penetration guidance strategy that balances both the guidance accuracy and penetration capability is proposed through deep reinforcement learning. Reward shaping and random initialization are applied to improve training speed and generalization, respectively. The proposed strategy is developed based on the twin delayed deep deterministic policy gradient algorithm. It directly maps observations to actions and is an end-to-end guidance scheme that does not require an accurate model. The simulation results show that the proposed strategy has higher penetration probabilities than conventional strategies for different initial heading errors and even for defenders with different guidance laws, which indicates its good robustness and generalization. For different initial heading errors, it has learned different maneuvering modes and has certain intelligence. In addition, it is computationally small, does not consume much memory, and can be easily applied on modern flight computers.