Native Long Video Extrapolation
Junhao Zhuang · Shiyi Zhang · Yuxuan Bian · Yaowei Li · Yawen Luo · Weiyang Jin · Songchun Zhang ·
Xianglong He · Xuying Zhang · Haoran Li · Haoyang Huang · Zeyue Xue · Nan Duan
Joy Future Academy, JD
📄 Paper 🐙 Code 🤗 Model
TL;DR — Self Gradient Forcing (SGF) recovers the missing context-gradient path for self-generated causal memory through a bounded two-pass replay, enabling models trained with only a 5-second window to extrapolate to minute-scale videos with stronger identity, layout, and temporal stability.
Our new work · SGF+ SGF+: Decoupling Gradient Flows for Autoregressive Video Generation Separating context writing and denoising, enabling up to 24-hour rollouts from only 5s training windows.
Abstract
SGF teaser: long-horizon consistency comparison

Recent autoregressive video diffusion methods are increasingly built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground-truth video contexts. This reduces exposure bias, but the historical key-value cache is still used by future frames only as frozen rollout state. As a result, future losses cannot supervise how earlier generated latents should be written into more useful keys and values for later video-latent generation. We call this the historical context-gradient gap. We propose Self Gradient Forcing (SGF), a two-pass training strategy that restores this missing supervision signal without backpropagating through the full serial rollout. Pass 1 performs a no-gradient autoregressive rollout matching inference and, at a sampled denoising exit step, records both the self-generated context and the noisy latents fed to the model. Pass 2 performs parallel context-gradient reconstruction for the recorded exit step: the generated context is used as stop-gradient clean-latent input, while the model recomputes the context KV representations and future-to-context causal attention. Thus, SGF provides the missing memory-writing supervision within the native autoregressive training objective, using losses on future video latents to train the model to encode context into more effective causal memory. Across extensive long-horizon frame-wise and chunk-wise experiments under different initializations, SGF achieves stronger native long-video extrapolation than Self Forcing, especially in subject identity, background/layout consistency, and temporal stability. Remarkably, using only a 5-second training window, SGF can extrapolate to videos lasting several minutes.

Method

Self Forcing trains on self-generated histories but treats historical K/V entries as detached cache state. SGF keeps the no-gradient rollout unchanged and adds a parallel reconstruction pass, where detached context latents are re-encoded at the clean context timestep so future DMD losses supervise K/V writing without serial rollout backpropagation.

SGF training pipeline flowchart
Figure 1: From frozen-cache Self Forcing to Self Gradient Forcing.
Chunk-wise Long-Horizon Generation

Four-way comparison per sample — SGF (Ours), Self Forcing (SF), SF + Bi-directional ODE, SF + Causal ODE. All four tiles play in lock-step; use the controls on the SGF (Ours) tile to play, pause, or scrub. Pick a duration, then switch the AR / CD initialization.

60s Chunk-wise Long Video

240s Chunk-wise Long Video

Frame-wise Long-Horizon Generation

Two-way lock-step comparison per sample: SGF (Ours) vs Self Forcing. The SGF (Ours) tile carries the video controls; the Self Forcing tile follows it in play, pause, and scrub. Pick a duration, then switch the AR / CD / ODE initialization.

60s Frame-wise Long Video

240s Frame-wise Long Video

Analysis
Table 1: Training feasibility of context-gradient recovery

Direct differentiable-cache training keeps the serial cache-formation graph and runs out of memory. SGF uses bounded parallel context-gradient reconstruction at the sampled exit step, restoring context K/V gradients with modest peak-memory and runtime overhead.

Training variant Peak memory Stable memory Time / 5 steps Outcome
Self Forcing, frozen historical K/V cache 79.01GB 79.01GB 10.39s trains
SGF, serial Pass 1 + parallel Pass 2 87.01GB 63.73GB 11.71s trains
Self Forcing, differentiable historical K/V cache OOM OOM -- OOM
Pass-1 / Pass-2 recovery fidelity

SGF relies on Pass 2 faithfully replaying the sampled exit computation from Pass 1. Figure 2 and Table 2 are the same recovery diagnostic: the decoded pairs are visually nearly indistinguishable, while the latent metrics show low relative L2 error and near-unity cosine similarity across exit steps.

Decoded Pass-1 and Pass-2 recovery comparison
Figure 2: Decoded Pass-1 and Pass-2 recovery comparison. Rows show paired decoded outputs from Pass 1 and Pass 2 at exit steps 1000, 750, 500, and 250; the decoded pairs are visually nearly indistinguishable, matching the latent-space recovery results in Table 2.
Table 2: Pass-1 versus Pass-2 latent recovery fidelity

Metrics are averaged over 24 prompts for each exit step. Overall averages are computed over all 96 prompt/exit comparisons. We also report the relative L2 error normalized by the bf16 relative precision εbf16 = 2-7.

Exit step # comparisons MSE RMSE Mean abs. Max abs. Rel. L2 Rel. L2 / εbf16 Cosine
1000 24 3.123e-4 0.01745 0.01093 0.58396 0.02133 2.73 0.999766
750 24 2.148e-4 0.01449 0.00884 0.58189 0.01566 2.00 0.999874
500 24 1.141e-4 0.01064 0.00693 0.49110 0.01125 1.44 0.999936
250 24 6.041e-5 0.00774 0.00535 0.29028 0.00812 1.04 0.999967
Overall 96 1.754e-4 0.01258 0.00801 0.48681 0.01409 1.80 0.999886
SGF in Practice

Recent systems that explicitly use Self Gradient Forcing in their causal training pipelines.

BibTeX
@article{zhuang2026self,
  title   = {Self Gradient Forcing: Native Long Video Extrapolation},
  author  = {Zhuang, Junhao and Zhang, Shiyi and Bian, Yuxuan
             and Li, Yaowei and Luo, Yawen and Liu, Yijun
             and Jin, Weiyang and Zhang, Songchun and He, Xianglong
             and Zhang, Xuying and others},
  journal = {arXiv preprint arXiv:2607.20368},
  year    = {2026}
}