Video generation models are increasingly pitched as world simulators that could support embodied planning and learning. But to serve that role, they must do more than produce convincing frames: they need to predict how a scene evolves as goal-directed actions are carried out. Existing benchmarks, according to the authors of a new paper, mostly cover single short actions or step-by-step instructions, leaving multi-step physical reasoning underexplored — particularly in egocentric video generation, where a model must plan a sequence of real-world manipulations to reach a high-level goal.
To address that gap, the researchers introduce Ego2Act, a goal-directed benchmark built from 2,640 videos covering 110 real-world tasks in day-to-day settings, with varying levels of object clutter and multi-step complexity. Given a single real egocentric start frame and a high-level goal — for example, "the round yellow paper is stapled to the white A4 paper" — a model must generate the entire manipulation sequence.
The benchmark draws on 660 human recordings (three correct and three incorrect per task) and 1,980 generations from six models across three seeds. When rated by humans, the strongest model, Seedance-2.0, averaged 64 out of 100, while the weakest scored below 10 out of 100.
The failure analysis is pointed: 88.7% of failed steps were either skipped or left half-done, leaving later steps without the dependent states they require and ultimately leaving goals unfulfilled. Physics also breaks down — objects morph, pass through solids, or open on impossible hinges.
To make evaluation scalable, the team also presents Ego2ActJudge, a reference-free pipeline that plans subgoals in order and scores both task completion and physics. It reaches a correlation of r = 0.69 with human consensus, compared with 0.40 for the best baseline and 0.76 for a human rater. The benchmark, code, and every generated video with its scores are released openly.