Optimizing the Location of Automated Power Warehouse Based on Multi-agent Reinforcement Learning

doi:10.3969/j.issn.1007-7375.2021.04.011

Abstract

Abstract: The optimization of the cargo location of an automated warehouse is vital to improve warehouse efficiency. Aiming at the optimization of power warehouse cargo location, the method based on multi-agent reinforcement learning is adopted to improve the optimization. First, the deficiencies of DDPG algorithm and MADDPG algorithm are analyzed, and on this basis an improved algorithm ECS-MADDPG and its model proposed. In this algorithm, both the immediate reward at the current time point and the future reward factors are considered. Finally, using the historical incoming and outgoing data of electric power materials, the reinforcement learning algorithm is applied to train the cargo location optimization model. Experiments show that ECS-MADDPG has higher stability and rewards compared with algorithms such as MADDPG and DDPG.

Key words: power warehouse, inventory optimization, multi-agent deep reinforcement learning

CLC Number:

TP13

WANG Tiezheng, HU Ya'nan, PAN Kun, YU Xiao. Optimizing the Location of Automated Power Warehouse Based on Multi-agent Reinforcement Learning[J]. Industrial Engineering Journal, 2021, 24(4): 93-99.

References

[1] YUE L, GUAN Z, HE C, et al. Slotting optimization of automated storage and retrieval system (AS/RS) for efficient delivery of parts in an assembly shop using genetic algorithm: a case Study[C/OL]. (2016-04-01). https://iopscience.iop.org/article/10.1088/1757-899X/215/1/012002.
[2] BOROVINEK M, EKREN B Y, BURINSKIEN A, et al. Multi-objective optimisation model of shuttle-based storage and retrieval system[J]. Transport, 2017, 32(2): 1-18
[3] HAMZAOUI M A, SARI Z. Optimal dimensions minimizing expected travel time of a single machine flow rack AS/RS[J]. Mechatronics, 2015, 31: 158-168
[4] 杨玮, 傅卫平, 王雯. 自动化立体仓库旋转货架拣货路径优化[J]. 机械科学与技术, 2012, 31(3): 424-428
YANG Wei, FU Weiping, WANG Wen. Optimization of picking path for rotating racks in automated warehouse[J]. Mechanical Science and Technology, 2012, 31(3): 424-428
[5] 印美, 洪荣晶, 刘林. 基于非支配遗传算法的自动化仓库动态货位优化[J]. 组合机床与自动化加工技术, 2015(3): 31-34
YIN Mei, HONG Rongjing, LIU Lin. Dynamic storage optimization of automated warehouse based on non-dominated genetic algorithm[J]. Modular Machine Tool and Automatic Manufacturing Technology, 2015(3): 31-34
[6] 张思建, 方彦军, 贺瑶, 等. 基于模拟退火算法的AVS/RS多批货箱入库货位优化[J]. 武汉大学学报(工学版), 2016, 49(2): 315-320
ZHANG Sijian, FANG Yanjun, HE Yao, et al. AVS/RS multi-batch storage location optimization based on simulated annealing algorithm[J]. Journal of Wuhan University (Engineering Edition), 2016, 49(2): 315-320
[7] 肖锋. 基于强化学习的库位优化算法在物料拉动系统中的研究与应用[D]. 成都: 西南交通大学, 2015.
XIAO Feng. Research and application of storage location optimization algorithm based on reinforcement learning in material pulling system[D]. Chengdu: Southwest Jiaotong University, 2015.
[8] MA Y, YUN W, HOU W. The research progress of genetic algorithm in the large warehouse system[C/OL]. (2011-01-13). https://ieeexplore.ieee.org/document/5662823.
[9] ATMACA E, OZTURK A. Defining order picking policy: a storage assignment model and a simulated annealing solution in AS/RS systems[J]. Applied Mathematical Modelling, 2013, 37(7): 5069-5079
[10] MUPPANI V R, ADIL G K. Efficient formation of storage classes for warehouse storage location assignment: a simulated annealing approach[J]. Omega, 2008, 36(4): 609-618
[11] CHEN Y, HE F. Research on particle swarm optimization in location assignment optimization[C/OL]. (2008-08-08). https://ieeexplore.ieee.org/document/4594428.
[12] SUTTON R S, BARTO A G. Reinforcement learning: an introduction[J]. IEEE Transactions on Neural Networks, 1998, 9(5): 1040-1054
[13] EGOROV M. Multi-agent deep reinforcement learning[DB/OL]. [2020-04-10]. http://cs231n.stanford.edu/reports/2016/pdfs/122_Report.pdf.
[14] 杜威, 丁世飞. 多智能体强化学习综述[J]. 计算机科学, 2019, 46(8): 1-8
DU Wei, DING Shifei. A review of multi-agent reinforcement learning[J]. Computer Science, 2019, 46(8): 1-8
[15] RYAN Lowe. Multi-agent actor-critic for mixed cooperative-competitive environments[C]. Advances in Neural Information Processing Systems. Long Beach, USA: Curran Associates, 2017: 6379-6390.
[16] 焦玉玲, 张鹏, 田广东, 等. 基于多种群遗传算法的自动化立体库货位优化[J]. 吉林大学学报(工学版), 2018, 48(5): 1398-1404
JIAO Yuling, ZHANG Peng, TIAN Guangdong, et al. Automated three-dimensional warehouse location optimization based on multi-population genetic algorithm[J]. Journal of Jilin University (Engineering and Technology Edition), 2018, 48(5): 1398-1404
[17] LITTMAN M L. Markov games as a framework for multi-agent reinforcement learning[C/OL]. (1994-07-10). https://doi.org/10.1016/B978-1-55860-335-6.50027-1.
[18] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning [DB/OL]. (2019-06-05). https://arxiv.org/abs/1509.02971.