基于门控循环单元强化学习的晶圆光刻区实时调度方法研究

doi:10.3969/j.issn.1007-7375.240100

工业工程 ›› 2024, Vol. 27 ›› Issue (3): 12-21,30.doi: 10.3969/j.issn.1007-7375.240100

• 晶圆制造过程优化与测试 • 上一篇下一篇

基于门控循环单元强化学习的晶圆光刻区实时调度方法研究

吴立辉¹, 石津铭¹, 金克山¹, 张洁²

1. 上海应用技术大学机械工程学院，上海 201400;
2. 东华大学人工智能研究院，上海 201620

收稿日期:2024-03-19 发布日期:2024-07-12
作者简介:吴立辉 (1981—)，男，湖南省人，副教授，博士，主要研究方向为智能制造、复杂制造系统调度
基金资助:
国家重点研发资助项目 (2022YFB3305003)；上海应用技术大学引进人才科研启动项目 (YJ2022-33)

Real-time Scheduling of Wafer Photolithography Area Based on Reinforcement Learning with Gated Recurrent Unit

WU Lihui¹, SHI Jinming¹, JIN Keshan¹, ZHANG Jie²

1. School of Mechanical Engineering, Shanghai Institute of Technology, Shanghai 201400, China;
2. Institute of Artificial Intelligence, Donghua University, Shanghai 201620, China

Received:2024-03-19 Published:2024-07-12

摘要/Abstract

摘要： 为求解具有动态性、实时性、多约束、多目标特点的晶圆光刻区调度问题，提出一种基于门控循环单元强化学习的晶圆光刻区实时调度方法。设计引入门控循环单元学习光刻区历史调度决策与状态的时序信息，为双深度强化学习模型提供辅助决策信息；设计双深度强化学习模型的输入状态空间、输出动作集，并面向晶圆最小化最大完工时间和晶圆准时交货率指标设计多目标奖励函数，为智能体优化调度输出；设计设备专用性约束与掩模版约束的解约束规则与调度方法相结合，提高调度方案实施的实用性。通过某晶圆制造企业实际算例，将该方法与传统双深度强化学习和光刻区启发式规则方法比较，该方法均为最优，证明了其解决此问题的有效性。

关键词: 晶圆制造系统, 光刻区调度, 深度强化学习, 门控循环单元(GRU), 多目标

Abstract: To address the scheduling problem of wafer photolithography area, characterized by dynamic nature, real-time requirements, multiple constraints, and multiple objectives, a real-time scheduling method based on gated recurrent unit (GRU) reinforcement learning is proposed. This method incorporates GRU to learn the temporal information of historical scheduling decisions and states in the photolithography area, providing auxiliary decision-making information for the double deep reinforcement learning (DDRL) model. The input state space and output action set of the DDRL model are designed, and a multi-objective reward function is established with the objective of minimizing the maximum completion time of wafers and maximizing the on-time delivery rate, optimizing the scheduling output by intelligent agents. Additionally, constraint relaxation rules and scheduling methods are proposed combining equipment-specific constraints and mask constraints, to enhance the practicality of scheduling strategies. Through empirical evaluation using real-world cases from a wafer manufacturing enterprise, this method is compared with traditional double deep reinforcement learning and heuristic rule methods for photolithography area, demonstrating its superiority and verifying its effectiveness in solving this problem.

Key words: wafer fabrication system, scheduling of photolithography area, deep reinforcement learning, gated recurrent unit (GRU), multi-objective

中图分类号:

F426.4
TP391

吴立辉, 石津铭, 金克山, 张洁. 基于门控循环单元强化学习的晶圆光刻区实时调度方法研究[J]. 工业工程, 2024, 27(3): 12-21,30.

WU Lihui, SHI Jinming, JIN Keshan, ZHANG Jie. Real-time Scheduling of Wafer Photolithography Area Based on Reinforcement Learning with Gated Recurrent Unit[J]. Industrial Engineering Journal, 2024, 27(3): 12-21,30.

参考文献

[1] CHEN H, GUO P, JIMENEZ J, et al. Unrelated parallel machine photolithography scheduling problem with dual resource constraints[J]. IEEE Transactions on Semiconductor Manufacturing, 2022, 36(1): 100-112
[2] LI X L, HUANG Y L, TAN Q, et al. Scheduling unrelated parallel batch processing machines with non-identical job sizes[J]. Computers & Operations Research, 2013, 40(12): 2983-2990
[3] MAECKER S, SHEN L, MÖNCH L. Unrelated parallel machine scheduling with eligibility constraints and delivery times to minimize total weighted tardiness[J]. Computers & Operations Research, 2023, 149: 105999
[4] HAM A. Scheduling of dual resource constrained lithography production: using CP and MIP/CP[J]. IEEE Transactions on Semiconductor Manufacturing, 2017, 31(1): 52-61
[5] 张朋, 张洁, 王卓君, 等. 基于分解多目标进化算法的光刻区调度方法[J]. 华中科技大学学报 (自然科学版), 2022, 50(4): 26-32
ZHANG Peng, ZHANG Jie, WANG Zhuojun, et al. Decomposition multi-objective evolutionary algorithm based photolithography area scheduling method[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022, 50(4): 26-32
[6] ZHANG P, ZHAO X, SHENG X, et al. An imperialist competitive algorithm incorporating remaining cycle time prediction for photolithography machines scheduling[J]. IEEE Access, 2018, 6: 66787-66797
[7] ZHANG Z, ZHENG L, LI N, et al. Minimizing mean weighted tardiness in unrelated parallel machine scheduling with reinforcement learning[J]. Computers & Operations Research, 2012, 39(7): 1315-1324
[8] ZHANG T, XIE S, ROSE O. Real-time batching in job shops based on simulation and reinforcement learning[C]//Proceedings of the 2018 Winter Simulation Conference (WSC) . Gothenburg: IEEE Press, 2018: 3331-3339.
[9] VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double q-learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016.
[10] MENG L, GORBET R, KULIĆ D. The effect of multi-step methods on overestimation in deep reinforcement learning[C]//Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR) . Milan: IEEE Press, 2021: 347-353.
[11] LIU R, PIPLANI R, TORO C. Deep reinforcement learning for dynamic scheduling of a flexible job shop[J]. International Journal of Production Research, 2022, 60(13): 4049-4069
[12] WANG M, ZHANG J, ZHANG P, et al. Independent double DQN-based multi-agent reinforcement learning approach for online two-stage hybrid flow shop scheduling with batch machines[J]. Journal of Manufacturing Systems, 2022, 65: 694-708
[13] LEE D, KANG H, LEE D, et al. Deep reinforcement learning-based scheduler on parallel dedicated machine scheduling problem towards minimizing total tardiness[J]. Sustainability, 2023, 15(4): 2920
[14] PAENG B, PARK I B, PARK J. Deep reinforcement learning for minimizing tardiness in parallel machine scheduling with sequence dependent family setups[J]. IEEE Access, 2021, 9: 101390-101401
[15] GAO S, HUANG Y, ZHANG S, et al. Short-term runoff prediction with GRU and LSTM networks without requiring time step optimization during sample generation[J]. Journal of Hydrology, 2020, 589: 125188
[16] ZHANG K, CHEN N, LIU J, et al. A GRU-based ensemble learning method for time-variant uncertain structural response analysis[J]. Computer Methods in Applied Mechanics and Engineering, 2022, 391: 114516
[17] OUKIL A, EL-BOURI A. Ranking dispatching rules in multi-objective dynamic flow shop scheduling: a multi-faceted perspective[J]. International Journal of Production Research, 2021, 59(2): 388-411

基于门控循环单元强化学习的晶圆光刻区实时调度方法研究

Real-time Scheduling of Wafer Photolithography Area Based on Reinforcement Learning with Gated Recurrent Unit

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张志伟, 李洛平, 杨晓英, 杨欣. 混合多策略MHSSA智能优化风电拉挤板生产排程[J]. 工业工程, 2024, 27(3): 114-129.
[2]	张萌, 孙璐璐, 苏兵, 王能民. 考虑空载损失的非集中式共同配送订单分派及路径优化研究[J]. 工业工程, 2024, 27(2): 107-118,137.
[3]	张中伟, 李艺, 高增恩, 武照云. 基于深度强化学习的柔性作业车间节能调度研究[J]. 工业工程, 2024, 27(1): 78-85,103.
[4]	周国华, 马依婷. 基于改进NSGA-II的铁路项目进度计划多目标优化[J]. 工业工程, 2023, 26(4): 85-95.
[5]	王书娟, 侯艳, 滕少华, 朱清华. 基于改进SPEA2的原油短期调度问题研究[J]. 工业工程, 2023, 26(3): 124-133.
[6]	汪和平, 龚星霖, 李艳. 基于改进NSGA-II算法的装配式建筑施工调度优化[J]. 工业工程, 2023, 26(2): 85-92.
[7]	李红岩, 杨晓英, 赵恒喆, 张志文. 轴承柔性智能制造的物料配送期量动态优化方法[J]. 工业工程, 2023, 26(2): 155-162.
[8]	熊浩, 郭昊颖, 鄢慧丽, 陈锦怡. 外卖配送路径多目标实时优化研究[J]. 工业工程, 2023, 26(1): 98-107.
[9]	唐红涛, 张缓. 基于绿色生产的混合流水车间调度问题研究[J]. 工业工程, 2022, 25(3): 115-123.
[10]	钱伟康, 唐红涛. 考虑恶化效应的多目标非置换流水车间调度研究[J]. 工业工程, 2022, 25(2): 128-136.
[11]	叶万余. 面向电力物联网URLLC业务的智能网络切片管理方法[J]. 工业工程, 2022, 25(1): 129-135.
[12]	曾强, 吴锦锦, 王培露, 李明金. 混合工作日历下批量生产柔性作业车间多目标调度方法[J]. 工业工程, 2021, 24(6): 8-17.
[13]	杨坤, 伏跃红, 江志斌. 基于分布式光伏发电的多目标分时电价优化策略[J]. 工业工程, 2021, 24(6): 108-115.
[14]	马跃如, 文铮, 易丹. 考虑老年人满意度的多目标居家养老护理员路径优化[J]. 工业工程, 2021, 24(1): 74-81,103.
[15]	刘雪红, 张涛, 彭兆, 王磊. 面向医疗器械企业灭菌工艺的不同容量平行机批调度[J]. 工业工程, 2021, 24(1): 82-89.