Researchers have introduced LeWAM, an end-to-end joint-embedding predictive architecture (JEPA) that combines world modeling with action generation for control. The work appears as a paper on Hugging Face.
Unlike typical JEPA pipelines that first train visual representations and action-conditioned dynamics before adding a separate control policy or solver, LeWAM trains everything together. Its design rests on two elements: a joint objective covering next latent state prediction and action flow matching, and an isotropic Gaussian constraint on the latent state. Together these allow a single model to execute policies directly and to plan efficiently by sampling actions and evaluating them with its learned latent dynamics.
The authors report substantial gains over prior JEPAs in visual planning. Success rates rise from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks, while planning runs 32.3 times faster. The team also highlights a per-plan time of about 19 ms.
Beyond planning, LeWAM is described as competitive with established methods on behavior cloning tasks in both simulation and on physical robots. The authors note the model is small — 17M parameters, trained on a single GPU with one hyperparameter — and that it works on a real robot, not only in simulation.
Interpretability analyses indicate that LeWAM learns control-relevant representations and action distributions that effectively guide test-time planning and control. Code, data, and checkpoints are released publicly.