Interactive video world models are crucial for long-horizon planning and exploration, but they suffer from compounding errors over time. Post-training methods like reinforcement learning (RL) can improve these models, yet they face a verification bottleneck: for arbitrary action sequences, there is no ground-truth future state to measure long-term drift.
The key insight behind WorldCycle is that reversible action cycles make verification possible. A sequence composed with its inverse must analytically return to the initial state, providing annotation-free supervision on long-horizon correctness. The framework constructs closed action cycles and their repeated executions from ordinary action sequences, optimizing two complementary rewards: a spatial closure reward that enforces symmetry between mirrored forward and reverse segments, and a temporal consistency reward that aligns states across repeated cycle executions.
These rewards force the model to learn actions as consistent state operators rather than memorized temporal patterns, and they extend naturally to out-of-distribution composite action cycles that the base model handles poorly. The authors also release CycleBench, a diagnostic benchmark for state-returning ability under complex action structures.
WorldCycle reduces state-returning drift by up to 44% and lifts composite-action accuracy nearly 4x over the base model, providing a vital foundation for physically grounded world models.