Online Scheduling Method for Hybrid Flow Shop with Batch Machines Based on Proximal Policy Optimization Algorithm

Batch processing machines enable continuous overlapping operations, which is important for shortening production cycle times, reducing unnecessary waiting times, and increasing productivity. However, when faced with dynamic shop-floor events, the batch processor's workpiece type machining selection leads to unavoidable variations in the completion time of each workpiece. Therefore, the focus of our research is to adaptively select the appropriate workpiece processing type for the batch processor based on the real-time shop floor production and machining characteristics in order to minimize the delay cost of all workpieces. A hybrid flow shop scheduling problem with batch processing machines is studied and modeled as a Markov Decision Process. Multiple real-time features of job-resource integration, which combine job processing information with workshop resource information, are designed. Furthermore, job selection rules and batch processing selection rules for batch processing machines are formulated. The intelligent body decides the workpieces to be machined by the machine and the type of workpieces to be batch processed based on the real-time characteristics of the decision points through the composite scheduling rules, constructs the reward payoff function of the intelligent body in terms of the cost of the workpieces' dragging time, and trains the network of the intelligent body through the proximal policy optimization algorithm. Numerical experiments were conducted on a large number of instances with different production configurations. The results confirm the superiority and generalizability of the proposed algorithm compared to heuristics.