Yang, Zhong, Dessouky and Postolache (2018) formulated the simultaneous scheduling of quay cranes (QCs), automated guided vehicles (AGVs), and yard cranes in an automated container terminal as a static, deterministic bi-level makespan-minimization model with AGV routing and a congestion-prevention rule, solved by a bi-level genetic algorithm (CPR-BGA). They explicitly left open the operationalization of this capability: real-time scheduling with dynamic path planning and true re-scheduling under on-line uncertainty (arriving ships, stochastic events), explicit assignment of containers to yard storage locations inside the optimization, better makespan/run-time algorithms, and faster (parallel/GPU) implementations. The literature shows that later work has advanced each ingredient separately — deep-reinforcement-learning methods for dynamic AGV-only scheduling, static integrated models that include container storage allocation, robust static multi-equipment scheduling under uncertain operation times, and parallel multi-AGV algorithms — but no known work delivers an on-line method that re-schedules the fully integrated QC–AGV–yard system, including AGV routing with congestion control and explicit container-to-storage-location assignment, at realistic scale within operational time budgets. The problem is to construct and validate such a method.

Background

In an automated container terminal (ACT), container ships are worked by quay cranes (QCs, the rail-mounted cranes at the berth that lift containers on and off ships), automated guided vehicles (AGVs, driverless vehicles that carry containers horizontally between the quay and the yard), and yard cranes — in the source paper, automated rail-mounted gantries (ARMGs), which stack containers into rectangular storage areas called blocks. Because these three equipment classes are tightly coupled (a QC cannot load a container until an AGV delivers it, and an AGV cannot finish until the ARMG is free), scheduling them separately produces delays and congestion, so research has moved to integrated models that decide QC sequences, AGV task assignments, AGV routes, and yard placements jointly. A standard objective is the makespan, T=max⁡k(Fk−Sk)T=\max_k(F_k-S_k), the elapsed ship-handling time from the first QC start to the last QC finish; minimizing it shortens vessel turnaround.

Yang, Zhong, Dessouky and Postolache (2018) built a bi-level programming model — an optimization model nested in two levels, where the upper level schedules QC–AGV–ARMG operations and the lower level selects AGV paths on the terminal road network — minimizing makespan while enforcing a congestion-prevention rule (at most two AGVs simultaneously on any auxiliary road). Since the resulting mixed-integer program is intractable at realistic size, they proposed the Congestion Prevention Rule-based Bi-level Genetic Algorithm (CPR-BGA, a genetic algorithm encoding scheduling decisions in upper-level chromosomes and AGV path priorities in lower-level chromosomes), and compared it with a rolling horizon procedure (RHP, a classical decomposition that re-solves a limited time window at a time). On small instances (4–24 containers) CPR-BGA ran in 1.76–3.46 s versus 119–935 s for RHP with an average makespan gap of 6.65%, and it produced feasible schedules for up to 2000 containers, though the 2000-container run took 833 s. The model is static and deterministic: the whole schedule is computed up front from one known ship plan, ARMG handling times are fixed constants, containers are identical 40-ft units, and yard placement is treated through simplified block-assignment variables rather than explicit container-to-storage-location decisions.

The paper's stated future work asks for the operational capability this model lacks, and its four items form one objective — turning the offline model into a real-time decision tool: (1) real-time scheduling strategies and decision-making algorithms with dynamic path planning under on-line uncertainty, enabling true re-scheduling as large ships and stochastic events arrive; (2) extending the integrated optimization with explicit assignment of containers to specific storage locations in the yard (detailed yard assignment) instead of the simplified variables; (3) more accurate or alternative heuristics and metaheuristics (comparisons against CPR-BGA) for better makespan/run-time trade-offs on large instances; and (4) GPU or other high-performance implementations so re-optimization is fast enough for busy terminals. Items (3) and (4) are means to the real-time objective rather than independent goals.

Later work introduced the main techniques now applied to this setting. Deep reinforcement learning (DRL) frames sequential decision-making as a Markov decision process (MDP) — states, actions, rewards — and learns a policy by interacting with a simulation; variants used in terminals include deep Q-networks (DQN) and multi-agent deep deterministic policy gradient (MADDPG). A digital twin is a real-time-synchronized virtual replica of the physical terminal used to train or run such policies. Proactive (robust) scheduling hedges against known uncertainty distributions up front, whereas reactive scheduling re-optimizes when disruption information arrives; the distinction matters because only reactive or learned-online methods can perform true real-time re-scheduling.

Problem Statement

Construct and validate a scheduling method for an automated container terminal that jointly achieves what Yang et al. (2018) posed as future work on their integrated QC–AGV–ARMG model. The method must: (i) decide, within one optimization, the quay-crane loading/unloading sequences, the assignment of transport tasks to AGVs together with their routes on the terminal network under the congestion-prevention constraint, the yard-side operations, and the explicit assignment of each container to a specific storage location in the yard (replacing the paper's simplified block-placement variables); (ii) operate on-line — rather than producing a single static up-front schedule, it must re-schedule dynamically as stochastic events arrive (large ships, equipment delays or breakdowns, variable handling and travel times), supporting dynamic path planning and true real-time re-scheduling; and (iii) return its decisions within operational time budgets for realistically large instances (hundreds to thousands of containers). The common objective binding these requirements is a real-time, integrated decision capability for the coupled quay–transport–yard system. Validation must quantify both solution quality (e.g., makespan or vessel-delay reduction against a static integrated baseline and/or real-time dispatching rules) and wall-clock decision latency under on-line event streams. The task fixes no absolute latency or quality threshold, and the operating point (instance sizes, re-optimization latency, and quality gap) is the solver's to establish and defend by measurement.

References

  1. Yang, Y., Zhong, M., Dessouky, Y., & Postolache, O. (2018). An integrated scheduling method for AGV routing in automated container terminals. Computers & Industrial Engineering, 126, 482–493. DOI 10.1016/j.cie.2018.10.007. https://doi.org/10.1016/j.cie.2018.10.007
  2. Yue, L., & Fan, H. (2022). Dynamic Scheduling and Path Planning of Automated Guided Vehicles in Automatic Container Terminal. IEEE/CAA Journal of Automatica Sinica, 9(11), 2005–2019. DOI 10.1109/JAS.2022.105950. https://doi.org/10.1109/JAS.2022.105950
  3. Zheng, X., Liang, C., Wang, Y., Shi, J., & Lim, G. (2022). Multi-AGV Dynamic Scheduling in an Automated Container Terminal: A Deep Reinforcement Learning Approach. Mathematics, 10(23), 4575. DOI 10.3390/math10234575. https://doi.org/10.3390/math10234575
  4. Li, S., Fan, L., & Jia, S. (2024). A hierarchical solution framework for dynamic and conflict-free AGV scheduling in an automated container terminal. Transportation Research Part C, 165, 104724. DOI 10.1016/j.trc.2024.104724. https://doi.org/10.1016/j.trc.2024.104724
  5. Gong, L., Huang, Z., Xiang, X., & Liu, X. (2024). Real-time AGV scheduling optimisation method with deep reinforcement learning for energy-efficiency in the container terminal yard. International Journal of Production Research, 62(21), 7722–7742. DOI 10.1080/00207543.2024.2325583. https://doi.org/10.1080/00207543.2024.2325583
  6. Jian, W., Zhu, J., & Zeng, Q. (2021). An Optimization Model of Integrated AGVs Scheduling and Container Storage Problems for Automated Container Terminal Considering Uncertainty. Symmetry, 13(10), 1904. DOI 10.3390/sym13101904. https://doi.org/10.3390/sym13101904
  7. Yue, L., Fan, H., & Fan, H. (2023). Blocks allocation and handling equipment scheduling in automatic container terminals. Transportation Research Part C, 153, 104228. DOI 10.1016/j.trc.2023.104228. https://doi.org/10.1016/j.trc.2023.104228
  8. Cai, L., Li, W., Zhou, B., Li, H., & Yang, Z. (2024). Robust multi-equipment scheduling for U-shaped container terminals concerning double-cycling mode and uncertain operation time with cascade effects. Transportation Research Part C, 158, 104447. DOI 10.1016/j.trc.2023.104447. https://doi.org/10.1016/j.trc.2023.104447
  9. Zhang, Y., Bao, X., Zhang, L., Chen, L., Tang, X., Zhang, Z., & Zheng, Y. (2023). Digital Twin Enhanced Reinforcement Learning for Integrated Scheduling in Automated Container Terminals. 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE), 1–6. DOI 10.1109/CASE56687.2023.10260497. https://doi.org/10.1109/CASE56687.2023.10260497
  10. Wang, Z., & Zeng, Q. (2022). A branch-and-bound approach for AGV dispatching and routing problems in automated container terminals. Computers & Industrial Engineering, 166, 107968. DOI 10.1016/j.cie.2022.107968. https://doi.org/10.1016/j.cie.2022.107968
  11. Yu, D., Hu, X., Liang, K., & Ying, J. (2022). A parallel algorithm for multi-AGV systems. Journal of Ambient Intelligence and Humanized Computing, 13(4), 2309–2323. DOI 10.1007/s12652-021-02987-3. https://doi.org/10.1007/s12652-021-02987-3
  12. Sun, P. Z. H., You, J., Qiu, S., Wu, E. Q., Xiong, P., Song, A., Zhang, H., & Lu, T. (2023). AGV-Based Vehicle Transportation in Automated Container Terminals: A Survey. IEEE Transactions on Intelligent Transportation Systems, 24(1), 341–356. DOI 10.1109/TITS.2022.3215776. https://doi.org/10.1109/TITS.2022.3215776
  13. Naeem, D., Gheith, M., & Eltawil, A. (2023). A comprehensive review and directions for future research on the integrated scheduling of quay cranes and automated guided vehicles and yard cranes in automated container terminals. Computers & Industrial Engineering, 179, 109149. DOI 10.1016/j.cie.2023.109149. https://doi.org/10.1016/j.cie.2023.109149
  14. Li, W., Cai, L., He, L., & Guo, W. (2024). Scheduling techniques for addressing uncertainties in container ports: A systematic literature review. Applied Soft Computing, 162, 111820. DOI 10.1016/j.asoc.2024.111820. https://doi.org/10.1016/j.asoc.2024.111820