Abstract: The current research on energy-efficient flexible job-shop scheduling problems (EFJSPs) cannot make full use of historical production data, and is insufficiently adaptable to the complex, dynamic and changeable job-shop production environment. In view of this, deep reinforcement learning is introduced to solve EFJSPs, where a representative method named deep Q-network (DQN) is utilized. First, EFJSP is transformed into a Markov decision process corresponding to reinforcement learning. Moreover, the state values characterizing the job-shop production states are extracted as inputs of a neural network. By fitting the state value function through the neural network, compound scheduling action rules are output to achieve the selection of workpieces and processing machines. Furthermore, scheduling action rules and reward functions are utilized to jointly optimize the total production energy consumption. Finally, solutions of the proposed method are compared with those using typical intelligent optimization algorithms, such as non-dominated sorting genetic algorithm, hyper-heuristic genetic algorithm and multi-objective wolf pack algorithm, in three cases with different scales. Results demonstrate the powerful search capability of DQN algorithm, and the distribution of optimal solutions is consistent with the optimization objective obtained by the proposed EJFSP model. These verify the effectiveness of the utilized DQN method.

Key words: energy-efficient flexible job-shop scheduling, deep reinforcement learning, deep Q-network, Markov decision