Deep reinforcement learning for optimization of spiral shaft design in shield machine

The increasing demand for highly customized shield machines imposes greater efficiency and accuracy requirements on the optimization. To address the inefficiencies and susceptibility to local optima in existing structural design methods, a novel approach to construct a simulation analysis surrogate model for creating an environment for the optimization of spiral shaft is introduced. It also improves the Deep Deterministic Policy Gradient (DDPG) algorithm to optimize the structural parameters. In the surrogate model, a Goal-oriented autoencoder (GOAE)-classifier model is employed to discriminate the feasibility of samples, and a hybrid surrogate model, utilizing a Self-attention Artificial Neural Network (Self-attention ANN) weight allocation mechanism, makes precise predictions for feasible samples. This model automatically assigns adaptive weights to sub-surrogate models. Within the DDPG framework, a novel serial-parallel hybrid structure for the Actor network is proposed, harnessing the specialized feature representation capabilities of multiple networks to enhance optimization policy accuracy. Additionally, an experience replay filtering mechanism based on sample similarity is introduced to ensure sample diversity and boost the performance of the optimization policy. A simulation analysis surrogate model is constructed on a generated dataset, and the improved DDPG algorithm is leveraged for the optimization of spiral shafts based on this surrogate model. Experimental results demonstrate that the constructed simulation analysis surrogate model facilitates rapid and precise analysis of spiral shaft, while the improved DDPG algorithm successfully optimizes spiral shaft structural parameters, ultimately improving spiral shaft performance.