Researchers have introduced USDCraft, a framework that reframes articulated 3D asset reconstruction as a programmatic modeling problem grounded in partial geometric evidence. The work targets real-to-sim robot manipulation, where policies trained in simulation must transfer to physical objects.
Existing mesh-based methods learn to infer articulation from annotated 3D assets, but the authors note that deployment becomes difficult when real-world objects fall outside the training distribution or when their meshes are incomplete or corrupted. USDCraft instead has a pretrained LLM write and revise executable programs for simulation-ready articulated assets, requiring no task-specific training.
Two components anchor the approach. Source geometry analysis converts the source mesh into a metric textual description that distinguishes observed surface from unknown space. Iterative geometric rechecking then re-encodes each candidate in the same representation, so discrepancies point to specific program edits while unobserved regions remain open to completion. Visual feedback and physical authoring guidance round out the modeling process.
The output consists of articulated USD assets with explicit physical properties that load into Isaac Sim without manual adjustment. According to the authors, experiments demonstrate leading articulation recovery on two benchmarks and validate USDCraft's effectiveness for real-to-sim-to-real robot manipulation.
The project page is available at xingyoujun.github.io/usdcraft, and the code is slated to be open-sourced within two weeks, pending release approval.