Recent advances in foundation models have enabled AI systems to automate research workflows, but most still rely on preprocessed text, code, or summaries, missing critical spatial, temporal, and procedural details. Hugging Face researchers address this gap with OmniScientist, an end-to-end omni-modal AI scientist that reasons directly from raw evidence across diverse modalities.
The system integrates a perception layer and three autonomous agents—for ideation, experimentation, and write-up—within a deterministic pipeline. This design allows observations to continuously shape research questions, experimental decisions, and final claims throughout the research lifecycle. Built-in code-based checks enforce novelty, statistical validity, execution provenance, and numerical traceability.
OmniScientist was evaluated on 36 real-data cases spanning five discipline families and four evidence types, including images, signals, audio, video, 3D structures, trajectories, tables, formulae, and graphs. It successfully completed the full path from raw data to a compiled manuscript in all 36 cases, achieving a mean overall paper score of 6.3 with the reference reasoning backbone.
In paired comparisons against a blind variant that received only precomputed scalar features, direct perception improved all seven evaluation dimensions and won 85% of head-to-head judgments. The authors conclude that lifecycle-wide perception is essential for evidence-grounded scientific discovery and offers a practical path toward broadly capable AI scientists.