An Online Scheduling Method for Hybrid Flow Shops with Batch Machines Based on Proximal Policy Optimization Algorithm

Batch processing machines enable continuous overlapping operations, which is important for shortening production cycle time, reducing unnecessary waiting time, and increasing productivity. However, when faced with dynamic shop-floor events, the selection of workpiece types for batch processing machines may lead to unavoidable variations in the completion time of each workpiece. To this end, our study focuses on adaptively selecting appropriate workpiece types for batch processing machines based on real-time shop floor production and machining characteristics to minimize the total delay cost of all workpieces. A hybrid flow shop scheduling problem with batch processing machines is studied and modeled as a Markov decision process. Multiple real-time features of workpiece resource are designed, which integrate job processing information with workshop resource information. Furthermore, job selection rules and batch processing selection rules for batch processing machines are formulated. An intelligent agent decides the workpieces to be processed by the machine and the type of workpieces to be batch processed based on real-time characteristics of decision points through the integrated scheduling rules, while a reward function based on total delay cost of workpieces is formulated to guide the decisions of the agent. The network of the agent is trained through the proximal policy optimization algorithm. Numerical experiments are conducted on a large number of instances with different production configurations. Results demonstrate the superiority and generalizability of the proposed algorithm compared to heuristic methods.