A Modular and Coordinated Multi-agent Framework for Flexible Job-Shop Scheduling Problems with Various Constraints

The Flexible Job-shop Scheduling Problem (FJSP) is essential in today's industrial manufacturing, as it can greatly enhance efficiency in production via real-time data processing. In FJSP, it is important to consider various constraints due to the complexity of real-world production environments. Traditional Meta-heuristic methods encounter challenges in accommodating intricate problem constraints and suffer from high computational complexity, while rule-based methods perform poorly. Single-agent deep reinforcement learning frameworks are not equipped to handle complex real-world production issues. On the other hand, although existing multi-agent deep reinforcement learning frameworks designed for FJSP can solve FJSP with various constraints through minor adjustments, they often lack coordination among agents, leading to inefficient performance and unstable training. In this paper, we design a modular and coordinated multi-agent Deep Reinforcement Learning (DRL) framework that can solve FJSP problems with various constraints by adding agents tailored to specific constraints. We introduce a novel multi-agent coordinated proximal policy optimization algorithm (MACPPO), which promotes cooperation among agents by achieving dynamic credit allocation. We conducted experiments on multiple-sized instances under various constraints including equipment calendars and transportation, to verify the superiority of our framework. Experimental results show the effectiveness of the proposed novel method in addressing both the original FJSP problem and the FJSP problem with transportation and equipment calendar constraints. This efficiency is achieved compared to other well-known scheduling approaches, demonstrating the flexibility and efficiency of the architecture we proposed under diverse constraints.