Autoregressive (AR) video diffusion models can in principle generate video for unbounded horizons, but they are trained on short clips (e.g., 5 seconds at 16 FPS), so rollouts beyond the training duration degrade through accumulated drift: over-saturated colors, inconsistent subjects, collapsed structure, and motion stagnation. Rolling Sink (Li et al., 2026) showed that this beyond-duration train-test gap can be substantially mitigated without any training by maintaining the key-value (KV) cache so that it approximately preserves three properties of its within-training-duration behavior: minimal drift, sliding time indices, and sliding semantics. The paper itself states that these properties are not an exhaustive characterization of within-duration cache behavior, that finite-horizon training leaves a residual mismatch that cannot be fully closed, and that multi-shot generation under time-varying prompts remains future work. The problem those statements pose is to complete the characterization of within-duration cache behavior, to design cache-maintenance mechanisms that further mitigate the residual drift of open-ended rollouts while preserving streaming efficiency, and to extend drift mitigation to multi-shot synthesis.

Background

Setting. Video diffusion models generate video by iteratively denoising noise with a diffusion transformer (DiT, the standard transformer backbone adapted for diffusion modeling). Autoregressive (AR) video diffusion models do this block by block: at each step a block of a few frames is denoised conditionally on previously generated blocks, so generation can continue indefinitely - the natural architecture for open-ended video generation, where length is not fixed in advance (as in a movie composed of shots ranging from seconds to minutes). Because attention over the whole history would be unbounded, each step instead attends to earlier content only through a cached window of key and value vectors (the KV cache, the same memory mechanism used in transformer language models), kept under a strictly bounded capacity for streaming efficiency.

The train-test gap. AR video diffusion models are trained on clips of fixed, short duration - the reference system, Self Forcing (Huang et al., 2025), uses 5-second clips at 16 FPS with a rolling KV cache. When testing within that duration, the cache holds clean, teacher-like context and quality is high; beyond it, the model conditions on its own imperfect outputs, errors compound, and the rollout drifts. This is a video instance of exposure bias, the classical mismatch between training on ground-truth context and testing on self-generated context. Self Forcing addresses the gap within the training duration; the Rolling Sink paper (Li et al., 2026) addresses what happens beyond it.

Rolling Sink and its cache properties. Li et al. (2026) observe that, since the prompt embedding is constant and each block's initial noise is drawn from the same Gaussian distribution, the conditioning cache is the main carrier of long-horizon drift. Their systematic analysis asks how the cache behaves within the training duration and distills three properties: first, minimally drifted - cached latents should be free of accumulated error; second, sliding indices - the cached blocks' time indices, encoded by rotary positional embeddings (RoPE, the standard positional encoding that tags each token with its position), should form a fixed-length sliding window on a global time axis; third, sliding semantics - the cached content should behave as a moving slice of an effectively endless video rather than a frozen prefix. Their training-free mechanism pins roughly 83 percent of the bounded cache as sink blocks whose RoPE indices are shifted forward and whose content is periodically rolled from within-duration history; this scales the 5-second-trained model to 5-30 minute rollouts with stable colors and consistent subjects, evaluated with VBench-Long (the long-video configuration of the VBench video-generation benchmark suite). The idea descends from attention sinks (Xiao et al., 2023): in language models, pinning the earliest tokens in the KV cache stabilizes streaming generation; LongLive (Yang et al., 2025), Rolling Forcing (Liu et al., 2025), and Deep Forcing (Yi et al., 2025) brought sink-based caches to AR video.

The open problem. The paper is explicit that this is only a baseline: the three properties 'should not be viewed as an exhaustive characterization of the actual within-duration behavior of the AR cache'; because training uses finite clips, 'we can never fully close this mismatch', so a residual can remain, and 'additional cache maintenance requirements or more advanced methods may further improve the open-ended synthesis at test time'. Moreover, the method targets single-shot generation under one fixed prompt, whereas general long video (e.g., movies) is multi-shot: a sequence of prompts that must introduce new semantics over time while preserving long-horizon stability, with coherent transitions between shots. No existing cache-maintenance method both characterizes the within-duration cache beyond these three properties and extends the mechanism to time-varying multi-shot prompts, and residual train-test drift is reduced but not eliminated or bounded; the characterization, the residual bound, and the multi-shot extension therefore remain open. The question below is what closing that residual gap - and extending the drift-mitigation principle to multi-shot synthesis - requires.

Problem Statement

Starting from an autoregressive video diffusion model trained on fixed finite-duration clips (e.g., Self Forcing, 5 seconds at 16 FPS) with a strictly bounded KV cache, the problem is to close, as far as achievable, the train-test gap that emerges when generation runs beyond the training duration. Because finite-horizon training leaves a residual mismatch that Li et al. (2026) state can never be fully closed, the achievable target is itself part of the question rather than an assumption. Following the open directions posed by Rolling Sink, the problem asks: (1) how to characterize the within-duration behavior of the AR cache more completely than the three properties approximately enforced by Rolling Sink - minimal drift, sliding indices, sliding semantics - identifying which additional cache-maintenance requirements or more advanced methods may further improve the open-ended synthesis at test time; (2) how to design cache-maintenance mechanisms (training-free or otherwise, provided streaming efficiency - bounded cache capacity and fixed per-step cost - is preserved) that build on this characterization and demonstrably reduce or eliminate residual long-horizon drift (color over-saturation, subject and structure inconsistency, motion stagnation) on rollouts well beyond the training duration, with whatever residual remains explicitly quantified; and (3) how to extend the same drift-mitigation principle to multi-shot AR video synthesis, where new prompts introduce new semantics over time, achieving coherent and smooth shot transitions while preserving long-horizon stability and visual fidelity beyond the limited training durations. These directions share one scientific objective - making open-ended AR video generation behave like within-duration generation under a bounded cache.

References

  1. Haodong Li, Shaoteng Liu, Zhe Lin, Manmohan Chandraker. Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion. arXiv:2602.07775 (2026). DOI: 10.48550/arXiv.2602.07775. https://arxiv.org/abs/2602.07775
  2. Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, Eli Shechtman. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. arXiv:2506.08009 (2025). DOI: 10.48550/arXiv.2506.08009. https://arxiv.org/abs/2506.08009
  3. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis. Efficient Streaming Language Models with Attention Sinks. arXiv:2309.17453 (2023). DOI: 10.48550/arXiv.2309.17453. https://arxiv.org/abs/2309.17453
  4. Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, Song Han, Yukang Chen. LongLive: Real-time Interactive Long Video Generation. arXiv:2509.22622 (2025). DOI: 10.48550/arXiv.2509.22622. https://arxiv.org/abs/2509.22622
  5. Kunhao Liu, Wenbo Hu, Jiale Xu, Ying Shan, Shijian Lu. Rolling Forcing: Autoregressive Long Video Diffusion in Real Time. arXiv:2509.25161 (2025). DOI: 10.48550/arXiv.2509.25161. https://arxiv.org/abs/2509.25161
  6. Jung Yi, Wooseok Jang, Paul Hyunbin Cho, Jisu Nam, Heeji Yoon, Seungryong Kim. Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compression. arXiv:2512.05081 (2025). DOI: 10.48550/arXiv.2512.05081. https://arxiv.org/abs/2512.05081
  7. Jia Li, Xiaomeng Fu, Xurui Peng, Weifeng Chen, Youwei Zheng, Tianyu Zhao, Jiexi Wang, Fangmin Chen, Xing Wang, Hayden Kwok-Hay So. Train Short, Inference Long: Training-free Horizon Extension for Autoregressive Video Generation. arXiv:2602.14027 (2026). DOI: 10.48550/arXiv.2602.14027. https://arxiv.org/abs/2602.14027
  8. Youngrae Kim, Qixin Hu, C.-C. Jay Kuo, Peter A. Beerel. MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens. arXiv:2603.12513 (2026). DOI: 10.48550/arXiv.2603.12513. https://arxiv.org/abs/2603.12513
  9. Yusuf Dalva, Pinar Yanardag. AdaState: Self-Evolving Anchors for Streaming Video Generation. arXiv:2605.30349 (2026). DOI: 10.48550/arXiv.2605.30349. https://arxiv.org/abs/2605.30349
  10. Jiahao Tian, Yiwei Wang, Gang Yu, Chi Zhang. Head Forcing: Long Autoregressive Video Generation via Head Heterogeneity. arXiv:2605.14487 (2026). DOI: 10.48550/arXiv.2605.14487. https://arxiv.org/abs/2605.14487
  11. Jiayu Chen, Junbei Tang, Wenbiao Zhao, Maoliang Li, Jiayi Luo, Zihao Zheng, Jiawei Yang, Guojie Luo, Xiang Chen. Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation. arXiv:2605.13111 (2026). DOI: 10.48550/arXiv.2605.13111. https://arxiv.org/abs/2605.13111
  12. Yu Lu, Junjie Yang, Piotr Koniusz, YuXin Song, Yi Yang. FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion. arXiv:2606.10671 (2026). DOI: 10.48550/arXiv.2606.10671. https://arxiv.org/abs/2606.10671
  13. Yu Meng, Xiangyang Luo, Letian Li, Wenyuan Jiang, Chen Gao, Xinlei Chen, Yong Li, Xiao-Ping Zhang. TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment. arXiv:2606.13035 (2026). DOI: 10.48550/arXiv.2606.13035. https://arxiv.org/abs/2606.13035
  14. Jiatong Li, Leo Liang, Linghe Kong, Yulun Zhang. FreqForcing: Autoregressive Long Video Generation via Spectral Self-Anchoring. arXiv:2607.27110 (2026). DOI: 10.48550/arXiv.2607.27110. https://arxiv.org/abs/2607.27110
  15. Shuwei Shi, Zhen Li, Muyao Niu, Chuanhao Li, Bo Zheng, Kaipeng Zhang, Yinqiang Zheng. Surprise Forcing: What to Remember, When to Skip in Long Video Generation. arXiv:2607.18436 (2026). DOI: 10.48550/arXiv.2607.18436. https://arxiv.org/abs/2607.18436
  16. Junhao Zhuang, Shiyi Zhang, Yuxuan Bian, Yaowei Li, Yawen Luo, Yijun Liu, Weiyang Jin, Songchun Zhang, Xianglong He, Xuying Zhang, Haoran Li, Haoyang Huang, Zeyue Xue, Nan Duan. Self Gradient Forcing: Native Long Video Extrapolation. arXiv:2607.20368 (2026). DOI: 10.48550/arXiv.2607.20368. https://arxiv.org/abs/2607.20368
  17. Yihao Meng, Zichen Liu, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng, Yue Yu, Hanlin Wang, Haobo Li, Jiapeng Zhu, Yanhong Zeng, Xing Zhu, Yujun Shen, Qifeng Chen, Huamin Qu. CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives. arXiv:2605.12496 (2026). DOI: 10.48550/arXiv.2605.12496. https://arxiv.org/abs/2605.12496
  18. Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, Ziwei Liu. VBench: Comprehensive Benchmark Suite for Video Generative Models. arXiv:2311.17982 (2023). DOI: 10.48550/arXiv.2311.17982. https://arxiv.org/abs/2311.17982