Transformer-based multi-agent reinforcement learning for flexible job shop scheduling with AGVs

Integrated scheduling in flexible job shops with AGV transportation constraints presents a complex optimization challenge due to the tight coupling between operation sequencing and material handling. Existing reinforcement learning methods, typically relying on recurrent or fully connected architectures, often struggle to capture global spatio-temporal dependencies and suffer from sequential computational bottlenecks. To address these limitations, this study proposes MA-Trans, a hierarchical deep reinforcement learning framework based on Proximal Policy Optimization and Transformer encoders. By designing a cooperative dual-agent architecture, the model jointly optimizes job scheduling and AGV dispatching, employing self-attention mechanisms to efficiently encode long-range interactions among manufacturing resources. Experimental results demonstrate that MA-Trans significantly outperforms state-of-the-art baselines, including ResGAT and LSTM-Ptr, in both makespan minimization and AGV load balancing. Beyond solution quality, the proposed Transformer-based architecture reduces training time by approximately 23.8% compared to recurrent baselines while maintaining low-latency inference of around 2.75 ms. Moreover, the model exhibits robust zero-shot generalization capabilities, adapting to varying problem scales and AGV configurations without retraining, thereby highlighting its potential for real-time smart manufacturing applications.