Real-time transaction scheduling in multiple-deep tier-to-tier shuttle-based storage and retrieval systems by using deep Q-learning

Multiple-deep shuttle-based storage and retrieval systems (MD-SBS/RSs) have gained significant attention due to their ability to achieve more compact storage by saving aisle space compared to single-deep systems. However, the additional relocation operations increase the complexity of the transaction scheduling problem. Tier-to-tier shuttle-based storage and retrieval systems (T-SBS/RSs) enable shuttles to travel across tiers rather than being restricted to specific rack tiers, providing improved operational flexibility and lower shuttle investment costs. Nonetheless, the added crossing-tier operations further complicate transaction scheduling. This study focuses on multiple-deep T-SBS/RSs (MDT-SBS/RSs) and addresses the challenge of real-time transaction scheduling under a dual-command cycle mode. An artificial intelligence method is utilized and the system is treated as an agent to learn an optimal real-time scheduling policy using a deep Q-network (DQN) for average cycle time reduction. To assess the performance of DQN for the proposed problem, benchmark cases are designed considering different configurations and relocation strategies. Experimental results demonstrate the superior performance of DQN compared to classical transaction scheduling policies such as Random, First-Come-First-Served, and Shortest Processing Time across different system configurations. This study contributes to enhancing the efficiency of MDT-SBS/RSs through intelligent scheduling policies, showing the potential for using deep reinforcement learning methods to improve operation outcomes in warehouse automation.