Researchers have introduced the World Embedding Benchmark, a new benchmark designed to study how video representations encode physical information. While physical fidelity has drawn growing attention in world models and video generation, the paper argues that how video representations actually capture physics remains poorly understood.
The benchmark comprises 8,000 controlled simulation cases drawn from 80 physical families, spanning fluid mechanics, solid mechanics, dynamics, and optics & electromagnetism. Each case pairs a rendered video with simulation-derived physical annotations, supporting three complementary tasks: text-video retrieval, physical-property regression, and multiple-choice video-description pair classification.
Using these tasks, the authors distinguish cross-modal physical alignment from the recoverability of quantitative physical information. Their evaluation shows that pre-trained omnimodal embedding models perform weakly on retrieval and near chance on within-family pair classification, while lightweight probes can still recover useful physical information from frozen video embeddings.
The study also finds a notable trade-off: continual contrastive training with physics-specific video-text pairs improves retrieval and pair classification but degrades physical-property regression, suggesting that alignment and quantitative recoverability pull in different directions.
Finally, the team applied the embeddings to retrieval-augmented generation with MiniMax-H3. Retrieved reference videos improved the physical fidelity of generated videos, with stronger retrieval models yielding larger gains. The physics-adapted LCO-Embedding-Omni achieved the strongest retrieval-augmented generation performance.
The authors plan to release all physics simulation engines, the complete benchmark dataset, evaluation code, and trained models and checkpoints to support future research on physically grounded representations and world models.