Deep reinforcement learning-based trajectory optimization for pressure vessel filament winding and analysis of multi-payout-eye co-layer hybrid winding

Filament winding trajectories govern component manufacturability and dictate mechanical properties via fiber distribution modulation. This study aims to develop more flexible and versatile trajectories by establishing a co-layer hybrid winding theory for multiple payout eyes and proposing a physics-guided deep reinforcement learning (DRL)-based optimization method. A physics-based custom filament winding simulation environment was built to evaluate various algorithms. Additionally, a comparative dome training-difficulty score was introduced to assess morphology effects under the tested geometry and parameters. Finally, filament winding trajectory experiments under different objectives were conducted using a custom-built multi-payout-eye filament winding machine, thereby verifying the feasibility of the reinforcement learning-based trajectory design. The results indicate that, compared to Twin Delayed Deep Deterministic Policy Gradient (TD3) and Proximal Policy Optimization (PPO) algorithms, the Soft Actor-Critic (SAC) algorithm exhibits the highest stability while maintaining high precision in the winding environment. DRL-based winding trajectories can achieve flexible and stable winding across different modes by employing a variable slippage coefficient. Within the tested geometry and parameter range, the score decreased with increasing aspect ratio, permitting greater offsets and design flexibility. Simultaneously, the winding of two distinct trajectories on the same helical layer is realized through the coordinated motion of dual payout eyes. This hybrid trajectory winding effectively mitigates fiber accumulation at the polar opening of the dome. Compared with the tangential path, the mean winding thicknesses of the adjacent and interlacing paths within the inner evaluation band were reduced by 43.9% and 50.6%, respectively.