Polygonal meshes remain the backbone of modern 3D pipelines, but generating high-quality, artist-style topology has been a bottleneck for interactive applications. Traditional autoregressive methods serialize meshes into token sequences, leading to slow inference and error accumulation. In a new paper, Hugging Face researchers present Meshy T2, a framework that leverages flow matching to generate native meshes directly, bypassing the limitations of token-based approaches.
At the core of Meshy T2 is a vertex-set mesh VAE that encodes a mesh into one continuous latent token per vertex. The decoder reconstructs vertices, edge connectivity, and face winding order in a single pass, preserving high-precision geometry and artist-authored topology without vertex quantization or welding. This design avoids the quality loss common in discrete tokenization.
Generation follows a coarse-to-fine cascade of two flow-matching models. First, an image-conditioned voxel flow sketches the overall shape as a coarse occupancy scaffold. Then, a mesh flow populates the scaffold with per-vertex latent tokens, conditioned on the image, the scaffold, and a requested vertex budget. This enables three practical capabilities: interactive generation speed through parallel synthesis, effective face-count control via the vertex budget, and native support for multi-part assets whose components emerge directly from the generated connectivity.
In experiments, Meshy T2 achieves state-of-the-art geometric fidelity and completes end-to-end image-to-mesh generation in a median of 6 seconds—over an order of magnitude faster than autoregressive baselines. The code and weights will be released on GitHub, making the framework accessible for further research and adoption.