Generating Scalable Vector Graphics (SVG) code from natural-language instructions is an open-ended task with no absolute visual ground truth, leaving both evaluation and policy optimization without a reliable signal. According to a new paper posted on Hugging Face, scalar metrics such as CLIP and Aesthetic scores, which are calibrated on natural images, transfer poorly to stylized vector content — and reusing them as reinforcement-learning rewards can trigger reward hacking.
To address both problems, the authors adopt rubric-based scoring. They first show empirically that prompting a vision-language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics do, both across samples and within individual instructions.
Building on this finding, they introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning). The method converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes. A judge VLM then scores rendered rollouts item by item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization.
Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On the MMSVG-Illustration and MMSVG-Icon benchmarks, it lifts the rubric score from 0.432/0.395 to 0.693/0.683, surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, according to the paper.
Ablations identify rubric design as the active lever for reinforcement learning on open-ended SVG generation. A project page is available, and code is slated for release on GitHub.