Hugging Face has released the technical report for MOSS-VL, an open vision-language model family designed with real-time interaction as a core capability. The models can perceive visual information while generating responses, a key feature for applications like live video understanding and interactive assistants.
MOSS-VL employs a gated cross-attention mechanism that allows the language decoder to attend to visual frames during generation, without including visual tokens in the decoded sequence. This design enables the model to continuously process incoming visual data while speaking, reducing the time-to-first-token (TTFT) latency.
The training process uses a synthesized interaction corpus to teach the model when to speak, stay silent, or revise its output, and a staged curriculum concentrates real-time-specific training in a final lightweight stage over a strong offline foundation. This approach yields two model variants: MOSS-VL-Instruct for offline tasks and MOSS-VL-Realtime for streaming.
In evaluations, MOSS-VL-Instruct is competitive at comparable scale and leads on temporal-reasoning video benchmarks. MOSS-VL-Realtime achieves the best average performance on three of four streaming benchmarks among open-source models, and sweeps all three proactive behavior subsets, scoring 66.0 vs. 37.5 for the best baseline on OmniMMI Proactive Alerting.
With 11.3B parameters, MOSS-VL widens its TTFT advantage over the same-backbone Qwen3-VL-8B from 2.8x to 5.1x as visual context grows. All five checkpoints, the training curriculum, and real-time inference code are publicly released on GitHub.