Text-guided image editing usually requires users to describe a change in words, but pinpointing the exact object or region can be difficult — particularly when several objects look alike. A new research paper, VibeEdit, proposes a different approach: let users draw spatial marks and attach optional short notes directly on the image.
These annotations together form a canvas instruction that specifies both where to edit and what to change. According to the paper, VibeEdit can perform object addition, removal, replacement, attribute modification, and movement without a separate text prompt.
To train the system, the authors constructed 1.55 million source-target edit pairs with object masks and structured edit descriptions, from which canvas instructions were rendered during training. The model adapts Qwen-Image-Edit using layer-decoupled conditioning that encodes source images and canvas instructions separately.
Training proceeds in two stages: region-weighted supervised fine-tuning, followed by rubric-guided reinforcement learning aimed at improving edit completion, local edit quality, and preservation of unedited areas.
On an independently built, human-curated benchmark of 419 cases focused on selecting targets among similar objects, VibeEdit achieved a VLM rubric score of 79.9 and an outside-region PSNR of 32.8 dB. The strongest text-instructed baseline in the evaluation, FireRed, scored 67.4 and 24.0 dB respectively.