Unified omni-modal models have made rapid progress in handling audio, vision, and language, but most existing benchmarks, training data, and learning methods still treat these modalities in isolation. According to a new paper posted on Hugging Face, this leaves audio-visual joint reasoning — where both streams are required to answer a question — poorly measured and under-developed.
To close that gap, the authors present three contributions. OmniReasoningBench is a benchmark of 1,150 multiple-choice and open-ended questions spanning two task types: reasoning over video and reasoning beyond video. The design ensures that audio and visual evidence are both indispensable to a correct answer.
The second piece is OmniQA, a data engine that automatically builds evidence-grounded question-answer pairs requiring cross-modal reasoning, along with time-stamped clue chains that guide annotation of the thinking process. Beyond the benchmark, OmniQA produces the training sets OmniReasoning-SFT-112K and OmniReasoning-RL-19K.
Finally, the paper proposes Modality-Factored Self-Distillation (MFSD), an on-policy self-distillation method. It scores each sampled response under modality-specific clue contexts, separating the contribution of individual clues from their cross-modal interactions to enable token-level credit assignment.
Combining this data and method, the resulting model OmniReasoning-30B-A3B reaches 50.0% on OmniVideoBench and 42.5% on OmniReasoningBench — improvements of 12.8 and 9.3 percentage points over the base model Qwen3-Omni-30B-A3B-Thinking. The authors also report substantial gains on general and long-video benchmarks, including Video-MME-v2, and frame the work as a foundation for future omni-modal joint reasoning research.