Hugging Face has highlighted a new research paper, Tetris3D, which tackles single-image 3D scene reconstruction with a focus on objects that are physically and geometrically coherent as a scene. The authors argue that existing methods tend to generate objects independently or couple them only implicitly, offering limited guidance for fine-grained spatial compatibility between neighboring objects that interact with one another.
To address this, Tetris3D explicitly conditions the generation of each object on the geometry of surrounding objects and their physical relationships. This guides each object's shape and pose to remain geometrically and physically plausible within the overall scene. According to the paper's TL;DR, the method reconstructs 3D scenes from a single image by generating objects in physical dependency order, using their neighbors as context so that shapes and poses fit together.
The work also introduces ComOb, described as a physics simulation-based dataset of 1.2 million scenes featuring physical interactions across diverse object categories, with per-object meshes and pairwise physical relation annotations.
The authors report comprehensive experiments on synthetic and real-world scenes showing that Tetris3D recovers coherent object shapes and poses even when interacting regions are occluded, and achieves state-of-the-art performance in both generation quality and physical stability.
A project page and GitHub repository are linked in the paper announcement, and the authors state that their code will be publicly released.