Visual generation is shifting from models invoked once per request toward agentic control processes that can plan, choose tools, inspect intermediate synthesized outputs, correct failures, and reuse experience from earlier tasks. In most current systems, the controller is a large language model (LLM) or vision-language model (VLM), while visual generation models act as tools or executors.
According to a new paper posted on Hugging Face, the field lacks a consistent standard for deciding when a generation system actually becomes agentic. Planning depth, tool use, multi-role collaboration, and reinforcement learning are frequently cited as evidence of agenticity, yet the authors argue none of these necessarily determines which generation decisions the controller is able to make.
The paper instead organizes the field by what the controller can directly control during generation. At L1 Conditioning Control, the controller prepares input to a predetermined generator but does not choose which visual operation runs. At L2 Execution Control, it selects and invokes actual generation, editing, rendering, or other content-modifying operations. At L3 Outcome-Adaptive Control, it observes an intermediate outcome and uses that observation to alter a later operation within the same task. At L4 Experience-Adaptive Control, it retains experience from completed tasks and applies it to decisions on future tasks. A separate L0 Fixed Support category covers generators, editors, evaluators, reward models, benchmarks, and fixed pipelines that have no deployed controller making generation-level decisions.
The authors stress that these levels describe a progressively broader decision-making scope, not model size, system complexity, output quality, tool or role count, or training method. Applying the framework across image, video, editing, 3D, world, slide, and user-interface generation, they trace how controller capabilities have evolved and how mechanisms are distributed across levels.
The paper also develops a level-conditioned evaluation framework that matches generators, tools, budgets, and evaluators across levels, aiming to isolate the value of broadening the controller's decision-making scope. It concludes by identifying key transitions: from generative models to executable control, to reliable outcome-driven revision, to reusable cross-task experience, and toward a future generator-as-controller regime.