Researchers behind a new paper on Hugging Face argue that generating runnable code is only the first step in visual creation. A program may execute without errors yet still violate the requested composition, appearance, or motion — a discrepancy the authors call the Program-to-Visual (P2V) gap.
To address it, they propose MaLiang-Harness, a unified framework that organizes MLLM-driven visual generation as a persistent process of construction, inspection, and revision. Its central design principle is that the evolving visual program, its construction history, and its verification all share a common revision reference.
The framework rests on three mechanisms: Persistent Executable Generation (PEG), which preserves programs and task context; Traceable Generation Process (TGP), which links edits to rendered evidence; and Revision-aware Editing and Verification (REV), which supports restoration and checks the current revision before completion. Together they coordinate planning, execution, and visual feedback across rendering backends.
In evaluation, the authors tested 11 powerful closed-source MLLMs on MaLiang-IBench and four on MaLiang-VBench, measuring generation success, visual quality, and computational cost. GPT-6-Astra reportedly achieved 100% generation success on both benchmarks, with 96.0% of image tasks and 76.9% of video tasks meeting all quality thresholds.
The comparison also surfaced a mismatch between general capability scores and visual generation performance, with similarly scored models differing substantially in their ability to satisfy visual requirements. The authors position MaLiang-Harness as a systematic basis for studying how MLLMs translate executable code into visual outcomes, while cautioning that general benchmarks may be poor predictors of this ability. The project is available at github.com/gulucaptain/MaLiang-Harness.