Researchers have introduced Multimodal Flow, a fully continuous generative model of language and vision that aims to sidestep the trade-offs of existing unified multimodal architectures. According to the paper, most current unified models either treat both language and quantized images as discrete tokens, or combine discrete language prediction with continuous image generation. The first approach introduces a visual quantization bottleneck, while the second requires modality-dependent objectives and sampling procedures.
Multimodal Flow instead adopts a unified continuous architecture that integrates multimodal continuous representations with a shared chunk-causal flow backbone. Text blocks and images are organized as ordered continuous hyperchunks, preserving textual token order and visual spatial structure. The backbone learns a single vector field over these hyperchunks through Flow Matching.
The design uses joint attention to enable cross-modal interaction, while modality-specific feed-forward networks process each modality separately. During training, the model predicts multiple target chunks in parallel; at inference, it generates hyperchunks sequentially.
The authors instantiate and pretrain a model called MF-1 on multimodal data. Across 0.6B, 1.2B, and 1.6B parameter scales, continued pretraining consistently improved multimodal modeling. With only 150B pretraining tokens, MF-1 achieved an average score of 82.8 across GenEval and DPG-Bench, and 75.3 across VQAv2, MMBench, and POPE — results the authors describe as competitive with unified models trained on substantially more data.
Under matched data, optimization, and parameter budgets, Multimodal Flow further outperformed representative hybrid and discrete models. The authors position continuous chunk-based embedding flow modeling as a new fully continuous paradigm for unified multimodal modeling, and have publicly released the related code and model.